Thursday, 13 February 2014

Analysis techniques for user facing applications

When performing analysis as preparation to writing test cases the tester will start by using general analysis techniques such as 'Reference, Inference, Conference'.

Using these techniques, the high level explicit and implicit test requirements will be identified that mainly cover business and application level use cases and scenarios. From this point the tester must look to drill down to identify more specific test requirements and conditions.

Cases and scenarios will break down into a multitude of paths through the application and the states it can be in along those paths, paths and states will break down further into specific data driving those states, the data will break down again into specific syntax.

Use Cases and Scenarios > Paths and States > Data > Syntax

or visit

It’s essential this decomposition of the test requirements and conditions is done so the various analysis and test case design techniques can be employed to best effect. The Development and Test teams have a large range of test case design techniques they can use, but here we'll just think about those used by the test team.

Black Box Techniques (Functional and Non-Functional)
The test team will be looking to design tests that mimic actions the user may perform, either individually or as a process. They will also want to test the way the system handles the data they use when performing these actions and the forms and structure this data could take.

Flow, State and Data
Test case design techniques can be broken down into certain types in line with what we aim to achieve by using them. They can focus on data the user may enter or the system may have stored for use, states and changes of states of the system or components of the system and the user's journey or flow through the system.

State Transition Testing (State)
The state of an application is the condition or mode that it is in. The state of the application remains static until an event occurs, at which point data is sent and a response, visible or otherwise to the user, is returned. This aspect of state transition within a system is the most critical for testing. The tester may not recognise a state change has occurred which may affect subsequent test cases.

State transition can occur for a number of reasons, including the user initiating the state change by interaction with the system, scripts being run by a service or a connected application being used.

The tester should consider:
  • what states components of the system may be in when initialised
  • transitions that can occur and what it takes to bring about that transition
  • combinations of states and how they may affect other components

Simple examples are those of a web site that uses Ajax and cookies.

- Cookies
When the web page loads for the first time it may call a script that creates a Cookie. This will be its default initial state and this state causes the local system to change. If the page is re-visited the initial state of the web page will now be different as the cookie exists. The cookie’s initial state however may change as it may be updated, if it’s expired the state of the system may change as the cookie is replaced or removed.

- Ajax
If the user then selects an option from a drop-down menu they will do so from the initial state of the drop-downs. Selecting an option in a drop-down menu changes the state of the web page and if the page uses Ajax, other elements on the page may be changed in response (visible to the user or not).

We’ll discuss next how we can use Equivalence Partitioning to reduce the huge number of possible test cases related to data. In a similar way we need to reduce the number of test cases it’s possible to identify, related to paths through the software, when we use state transition testing as a test case design technique.

As it isn't possible to test every state and path the tester needs to consider coverage and the most important states which test:

  • paths that exercise the most state-to-state transitions
  • error and error recovery states
  • the least common paths between states

Equivalence Partitioning (Data)
A major issue faced by the tester is how to minimise the number of test cases while covering as many valid test conditions as possible, especially where these cases cover functionality that can accept a large range of data.

It’s likely that where a large range of data can be entered into or used by the application under test, this will be a large range of algorithmically similar but syntactically different data. An example may be postcodes and addresses or bank account numbers and balances. Within this overall data set there will be many equivalent groups.

An example of this is where a field may take currency values between £0.01 and £100.00 (one pence and one hundred pounds). We could separate this into ranges of ‘£0.01 to £0.99’ and ‘£1.00 to £100.00’ and say any value in these two groups is essentially equivalent. Those values being either a whole pence or whole pound values, in which case we may test with data of £0.50 and £50.00. If test cases with this data pass we could assume any other case with data sitting within the Equivalence Partition would also pass.

In this event data can be partitioned into equivalent groups and a reduced number of test cases written to cover the data. This will result in more efficient testing as a valid and representative sample of the whole data set is being tested.

Boundary Value Analysis (Data)
Where data that can be entered or used by the system is within specified ranges we can say that the lowest and highest ranges sit on a boundary. Similarly where we’ve split the data range into equivalent groups the lowest and highest values in those groups also sit on boundaries.

Assessing where these boundaries sit within the data range is the purpose of Boundary Value Analysis. The outcome of doing this analysis is to identify possible test cases that may be used to find less than robust coding around these boundaries.

Examples of the types of errors that can be found using this design technique include use of incorrect logical operators or stating invalid data ranges as acceptable for entry or use.

Syntax Testing
Something that seems to get forgotten when working with data, is whether that data is syntactically correctOnce we know what type of data is allowable and we’ve used it to test ranges and boundaries are valid and have been defined and coded in a robust way, our next step is to test the data itself.

For the purposes of our discussion, data can be considered to have two attributes:
    • Type
    • Syntax

  • Data can be side to be in one of two conditions:
    • Valid
    • Invalid

Examples could be:
  • Insufficient numbers in an account field
    • valid type of data
    • invalid form of syntax
  • A string of extended characters in an address field
    • Invalid type of data
    • invalid form of syntax

Controlling the scope of Test Cases
When we use test case design techniques to identify useful test cases, we should consider the need to limit the scope of the test case.

This generally means using a test case to test for a specific condition or failure mode. For example, we wouldn’t look to test a boundary that’s likely to fail in combination with data that’s also likely to fail. The idea of ‘1 Test Case, 1 Test’ comes into play here.

In practice however it may be more efficient to have a test case that could fail in many ways. An example could be a test case that has invalid data that should be trapped, on a boundary that is out of limits. We would observe the data validation failure and once fixed, re-run the test case to fail on the out of bounds condition.

This is often how automation test cases are written in order to hit the application hard in as broader way as possible.

The merits of multiple or single test conditions, within test cases being executed manually or with automation, during the progressive or regression testing is open for discussion and as always - driven by the context of the organisation.


or visit