When performing analysis as preparation to writing test cases the tester will start by using general analysis techniques such as 'Reference, Inference, Conference'.
Using these techniques, the high level explicit and implicit test
requirements will be identified that mainly cover business and application level use cases and scenarios. From
this point the tester must look to drill down to identify more
specific test requirements and conditions.
Cases and scenarios will break down into a multitude of paths
through the application and the states it can be in along those paths, paths
and states will break down further into specific data driving those states, the
data will break down again into specific syntax.
Use Cases and Scenarios > Paths
and States > Data > Syntax
or visit
It’s essential this decomposition of the test requirements and
conditions is done so the various analysis and test case design techniques can
be employed to best effect. The Development and Test teams have a large range
of test case design techniques they can use, but here we'll just think about those used by the test team.
Black Box Techniques (Functional
and Non-Functional)
The test team will be looking to design tests that mimic actions
the user may perform, either individually or as a process. They will also want to
test the way the system handles the data they use when performing these actions
and the forms and structure this data could take.
Flow, State and Data
Test case design techniques can be broken down into certain types
in line with what we aim to achieve by using them. They can focus on data the user may enter or the system may have stored for use, states and changes of
states of the system or components of the system and the user's journey or flow
through the system.
State Transition Testing
(State)
The state of an application is the condition or mode that it is
in. The state of the application remains static until an event occurs, at which
point data is sent and a response, visible or otherwise to the user, is
returned. This aspect of state transition within a system is the most critical
for testing. The tester may not recognise a state change has occurred which may
affect subsequent test cases.
State transition can occur for a number of reasons, including the
user initiating the state change by interaction with the system, scripts being
run by a service or a connected application being used.
The tester should consider:
- what states components of the
system may be in when initialised
- transitions that can occur and
what it takes to bring about that transition
- combinations of states and how
they may affect other components
Simple examples are those of a web site that uses Ajax and
cookies.
- Cookies
When the web page loads for the first time it may call a script
that creates a Cookie. This will be its default initial state and this state
causes the local system to change. If the page is re-visited the initial state
of the web page will now be different as the cookie exists. The cookie’s
initial state however may change as it may be updated, if it’s expired the
state of the system may change as the cookie is replaced or removed.
- Ajax
If the user then selects an option from a drop-down menu they will
do so from the initial state of the drop-downs. Selecting an option in a
drop-down menu changes the state of the web page and if the page uses Ajax, other elements on the page may be changed in response (visible to the user or not).
We’ll discuss next how we can use Equivalence Partitioning to
reduce the huge number of possible test cases related to data. In a similar way
we need to reduce the number of test cases it’s possible to identify, related
to paths through the software, when we use state transition testing as a test case design technique.
As it isn't possible to test every state and path the tester needs to consider coverage and the most important states which test:
- paths that exercise the most
state-to-state transitions
- error and error recovery states
- the least common paths between
states
Equivalence Partitioning
(Data)
A major issue faced by the tester is how to minimise the number
of test cases while covering as many valid test conditions as possible, especially
where these cases cover functionality that can accept a large range of data.
It’s likely that where a large range of data can be entered into
or used by the application under test, this will be a large range of
algorithmically similar but syntactically different data. An example may be
postcodes and addresses or bank account numbers and balances. Within this
overall data set there will be many equivalent groups.
An example of this is where a field may take currency values
between £0.01 and £100.00 (one pence and one hundred pounds). We could separate
this into ranges of ‘£0.01 to £0.99’ and ‘£1.00 to £100.00’ and say any value
in these two groups is essentially equivalent. Those values being either a
whole pence or whole pound values, in which case we may test with data of £0.50
and £50.00. If test cases with this data pass we could assume any other case
with data sitting within the Equivalence Partition would also pass.
In this event data can be
partitioned into equivalent groups and a reduced number of test cases written
to cover the data. This will result in more efficient testing as a valid and
representative sample of the whole data set is being tested.
Boundary Value Analysis
(Data)
Where data that can be entered or used by the system is within
specified ranges we can say that the lowest and highest ranges sit on a
boundary. Similarly where we’ve split the data range into equivalent groups the
lowest and highest values in those groups also sit on boundaries.
Assessing where these boundaries sit within the data range is the purpose
of Boundary Value Analysis. The outcome of doing this analysis is to identify
possible test cases that may be used to find less than robust coding around
these boundaries.
Examples of the types of errors that can be found using this
design technique include use of incorrect logical operators or stating invalid
data ranges as acceptable for entry or use.
Syntax Testing
Something that seems to get forgotten when working with data, is whether that data is syntactically correct. Once we know what type of data is allowable and we’ve used it to
test ranges and boundaries are valid and have been defined and coded in a
robust way, our next step is to test the data itself.
For the purposes of our discussion, data can be considered to have two
attributes:
- Data can be side to be in one of
two conditions:
Examples could be:
- Insufficient numbers in an account
field
- valid type of data
- invalid form
of syntax
- A string of extended characters in
an address field
- Invalid type of data
- invalid
form of syntax
Controlling the scope of
Test Cases
When we use test case design techniques to identify useful test cases, we should consider the need to limit the scope of the test case.
This generally means using a test case to test for a specific
condition or failure mode. For example, we wouldn’t look to test a boundary
that’s likely to fail in combination with data that’s also likely to fail. The
idea of ‘1 Test Case, 1 Test’ comes into play here.
In practice however it may be more efficient to have a test case
that could fail in many ways. An example could be a test case that has invalid
data that should be trapped, on a boundary that is out of limits. We would
observe the data validation failure and once fixed, re-run the test case to
fail on the out of bounds condition.
This is often how automation test cases are written in order to
hit the application hard in as broader way as possible.
The merits of multiple or single test conditions, within test cases being executed manually or with automation, during the progressive or
regression testing is open for discussion and as always - driven by the context
of the organisation.
Mark.
or visit
