Re-writing Performance Non-functional Requirements using a BDD Approach

At my latest client site I was recently tasked with reviewing the Performance testing NFRs (non-functional requirements) and asked to make them both more readable and more testable. The client gave me a huge list of requirements that were quite painful to read. Here is an example of a couple of them:

Performance requirement 1:

The My Account Details functionality shall contains member’s account details and Members Digital Card and display relevant result(when connected to a web service) to the customer within a time of 2 seconds when accessed.

Performance Requirement 2:

The My Account Details functionality shall contains member’s account details and Members Digital Card and display relevant result (when not connected to a web service) to the customer within a time of 4 seconds when accessed.

 

There were many of these requirements written in a very similar form. The first thing that I did with the above was combine it into a single requirement, that would wait for the maximum of 4 seconds. The typical end-user has no idea whether the application is connected to a web service or not, and the non-functional requirements should specify WHAT something does, not HOW it should do it.

Next I decided that this would be a good opportunity to introduce a BDD approach to the client. They had expressed a great interest in implementing BDD in their testing, so this seemed a good place to start. I re-wrote the above two requirements in a single user story following the traditional GIVEN / WHEN / THEN pattern. The new requirement read like this:

 

Scenario: User can access 'My account' details page in a timely fashion

Given: The user has logged into the app
When: The user accesses the My account details page
Then: The page will display the member's account details within 4 seconds

 

This is so much clearer and easier to read than the original requirements, and the client were overwhelmingly positive about the new approach. When I first did some research into using BDD scenario for performance NFRs I was a bit skeptical, but after writing out a couple of them I could see straight away how well they fitted for this situation.

We could further improve the scenario above to include some volumetric information, which is highly important to have when defining performance NFRs for your system:

 

Scenario: User can access 'My account' details page in a timely fashion

Given: The user has logged into the app
    and: It is the peak hour of usage
    and: There are 1000 concurrent users logged into the system
When: The user accesses the My account details page
Then: The page will display the member's account details within 4 seconds

 

Any time that I have to write lots of performance NFRs in the future, this will be the approach that I will try to adopt. Admittedly it wouldn’t work for other types of performance NFR, such as server resource metrics staying within a certain range, but for the performance of the system from the point of view of an end-user this seems like a great approach

  • Pingback: Testing Bits – 7/12/15 – 7/18/15 | Testing Curator Blog()

  • Gabriel Oliveira

    Hi James !

    It`s nice to see how readable the requirement became ! However, in your example, you hid from the reader (both, final users and the team) the need to pay attention to those devilish details (with WS connection and without).

    Shouldn`t it be more explicitly represented ? My concern is that one may easilly hide the complexity of a scenario from the reader with that simplified description, wich in turn may cause misunderstandings and confusion.

    Best regards!

    • http://james-willett.com James Willett

      Hi Gabriel,

      Thank you for your comment. You are absolutely correct that I hid the detail about the Web Service from both the final users and team, and that was my intention.

      In this case, the “team” consists of business analysts, project managers, key stakeholders, senior managers (as well as testers and developers). A lot of the team members who are looking at (and also approving) these requirements will not even know what a web service really is (and don’t really need to know, it’s a technical detail). All that the people approving this document really care about is, “will the performance be under 4 seconds?” – they don’t need or want to know the technical detail behind how it works, just that it does work.

      It might be the case that the technical details of using / not using web services be recorded in some other documentation specifically for the developers / testers, but the Performance NFR document (which is viewed by a wide range of people in the business) should not contain the specific technical details – it should say WHAT something does, not HOW it does it.

      Hope that clarifies it for you.

      Cheers,

      James

  • Barry Perez

    Hi James,

    A great and fresh approach! NFRs are seldom good (if they even exist) and if it comes down to us as the performance tester to document the requirements then this could be a useful way to ensure they’re captured in a consistent and readable manner!

    Keep up the good work!

    Barry

    • James Willett

      Thanks Barry! Agree, it can be a nightmare to get hold of any solid NFRs from the business. I have also found that the NFRs you get can very much be in the “Lick your finger and stick it in the air” , but at least that is something….

      With no NFRs at all, how do you know if your test is passing or not?? 🙂