Thursday, 13 February 2014

Test Approaches - Ad-Hoc, Random, Exploratory, Scripted, Regression


When considering how to structure the testing of a piece of software, the tester first needs to decide on the most appropriate approach that should be used. In truth it is most likely a combination of approaches.


In thinking of how to model these in a relational way, we can consider there to be five general approaches;


Ad-Hoc  /  Random  /  Exploratory  /  Scripted  /  Regression


To state it again, in practice the approaches will most likely be combined and this should be decided at the analysis and planning phase.


YouTube Channel: [WATCH, RATE, SUBSCRIBE]
or visit

Ad-Hoc
The purpose of the Ad-Hoc testing approach is to allow the test team to address any area and any aspect of the software, as they see fit, in an entirely free-form manner during a test session.

The Ad-Hoc test approach is generally focused entirely on finding bugs and does not expend the team’s time on planning the test activity, recording analysis of test requirements, writing test cases, etc. in advance of the test session or on applying any particular test techniques or knowledge of the system other than their own know how.

When the Ad-Hoc tests find issues, they are recorded as bugs as per the usual approach when bugs are found in testing. It’s common for a bug discovered with Ad-Hoc testing to be used as the basis for a new test case, assuming no test case already exists to cover the test path the bug was found on.


Random

A Random test approach allows the test team to execute a subset of test cases or test conditions. The subset should be representative of the whole so that the outcome will be representative of the outcome expected of the entire set.

In order to conduct Random testing there for an agreed set of test conditions and/or test cases need to be in place so a valid subset can be selected for execution.

Random testing can be of particular use where:

  • a large volume of existing test cases or test conditions could be run
  • the set can’t easily be prioritised or can’t be prioritised in any significant way
  • the organisation wishes to minimise the risk of selecting a biased subset

This is particularly common where there are very large data sets or numbers of records such as in data or record migration. In this case selecting a valid size for the subset may be done using a defined model such as that presented in ISO 2859.

As you've guessed no doubt, Ad-Hoc is not the same as Random.


Exploratory

Whenever a tester is following any testing approach and executing any type of test or test condition and then follows an unscripted or unplanned path prompted by their observations – they are following an Exploratory approach.

The Exploratory approach has at its core the idea that:

  • The skill, knowledge and experience of the tester, relating to both testing and the product, are the key to finding the most important bugs.
  • Test cases are simultaneously decided on and executed as ongoing discovery of software behaviour and learning about the software progresses.

To help focus effort a time-boxed Testing Session can be agreed along with a defined Objective or Charter for the testing. This way the tester knows how long they have and for what reason they are testing. A test lead can easily set a number of Objectives per area of the software and assign a tester to it for a time-boxed period.

The output of the testing session is of course bug reports, however there may also be written notes that can be reviewed in periodic catch-up meetings and which may form the basis for new scripted test cases.

Exploratory testing can also be run completely free-form with no Objectives defined or specific area of the software assigned for testing. This may appear to be more like an Ad-hoc approach however the major difference is in the application of the skill, experience and structured thinking of the tester.

Scripted
The most commonly used approach to delivering testing is the Scripted approach. This sees the team producing a set of test objectives and/or test cases that prescribe the testing to be conducted in very specific terms.

Fully scripted test cases are preceded by a phase of analysis in which the cases are produced. The main expectation is that test cases will not be written ‘on the fly’ during testing or that test execution will wander off the prescribed testing path.

In practice this strict adherence to just running the test cases rarely happens, except by the most junior members of the test team. As outlined in the Exploratory approach above testers will often find themselves performing a combination of Scripted and Exploratory testing at certain points whether intended or not.

Regression
Regression testing ensures that changes to existing code have not introduced bugs in code proven to work in earlier test runs. In addition a sub-set of the regression pack can be run to allow more targeted testing. An example of this would be running just the integration test cases.

When test cases have been executed and have passed they remain of use as they can be included in a Regression pack. Typically the test cases that cover the most critical functionality or the broadest set of functionality will be added to the Regression pack.

Where sets of test cases within the Regression pack are frequently used these are typically marked out as candidates for converting to automated scripts to further improve the efficiency of running them.

Confidence or Bug Finding?
Two aspects that will need to be considered when deciding on which approaches to use are:

  • The organisations focus on ‘Bug Finding’ compared to ‘Confidence’ in the status of the application
  • The type of application under test
In the diagram below we can see the approaches placed onto an analogue scale that ranges between having a focus on ensuring Confidence across to a greater focus on Bug Finding.





This is useful to help the organisation understand how the selection of approaches or combination of Approaches can support the needs they have of the testing effort.

For example, an entirely new piece of development would need Scripted testing to ensure all new functionality was covered. However this should be combined with Exploratory testing to ensure there was an appropriate focus on finding bugs.

Mark.


YouTube Channel: [WATCH, RATE, SUBSCRIBE]
or visit

0 comments: