Avoiding the NSA, GCHQ - Part 1

Can we avoid being spied on?

Don't get too close to the team?

Why it's GOOD to get close to the team.

Test Case execution priority?

Getting the priority right is critical to the focus of the testing effort.

What is performance testing?

Breaking Performance Testing down into sensible steps.

Ask questions

If you don't ask, you won't learn. You won't teach either.

Thursday, 13 February 2014

Overview of basic analysis techniques

In order to write the test case sets the test group must perform analysis of available information and of the Test Item if it’s also available.

This analysis is intended to identify the overall Test Requirements that exist and break these down into the required test case sets.


Reference
Refer directly to the available information such as design and technical documentation and the test item itself.

Inference
Apply domain knowledge, previous experience with similar test items and understanding of customer needs to infer appropriate test cases.

Conference
Speak directly to developers, stakeholders and customers to elicit valid test cases.


In addition to the above approaches members of the test group will typically use:
  • Inductive analysis to identify test Cases before execution
  • Deductive analysis to add additional test cases once bugs are found


Image source http://toknow-11.wikispaces.com/Inductive+Reasoning


When we apply an Inductive approach we work from the perspective that a bug is present in the test item and try to evaluate how it will manifest itself in the form of erroneous behaviours and characteristics of features or functionality.

This may be validation not covering boundaries correctly or style tags not working on a particular browser. We need to move from the specific bug that we assume or know is present and ascertain the broader scope of its effect on potentially numerous areas of the test item.

With Deductive analysis we assume that erroneous behaviours and characteristics of features or functionality have been observed and we now need to work backwards to the bug which is the cause.

An example might be where style and layout on multiple pages of a website is not as expected, the specific cause perhaps being a bug in a CSS file. In this way we attempt to move from the general to the specific. It may be that a range of issues have a single cause and the test group assessing this will help development deliver fixes more efficiently.

The test group will naturally apply Inductive and Deductive analysis as they are testing. For example, when a bug is found thought will be given to what other functionality may be affected and these areas of functionality will be checked.

When errors are observed that seem similar in nature connected paths of the user journey may be tested to see if these lead back to a single cause.

Mark.


YouTube Channel: [WATCH, RATE, SUBSCRIBE]
or visit


Read More






Test Approaches - Ad-Hoc, Random, Exploratory, Scripted, Regression


When considering how to structure the testing of a piece of software, the tester first needs to decide on the most appropriate approach that should be used. In truth it is most likely a combination of approaches.


In thinking of how to model these in a relational way, we can consider there to be five general approaches;


Ad-Hoc  /  Random  /  Exploratory  /  Scripted  /  Regression


To state it again, in practice the approaches will most likely be combined and this should be decided at the analysis and planning phase.


YouTube Channel: [WATCH, RATE, SUBSCRIBE]
or visit

Ad-Hoc
The purpose of the Ad-Hoc testing approach is to allow the test team to address any area and any aspect of the software, as they see fit, in an entirely free-form manner during a test session.

The Ad-Hoc test approach is generally focused entirely on finding bugs and does not expend the team’s time on planning the test activity, recording analysis of test requirements, writing test cases, etc. in advance of the test session or on applying any particular test techniques or knowledge of the system other than their own know how.

When the Ad-Hoc tests find issues, they are recorded as bugs as per the usual approach when bugs are found in testing. It’s common for a bug discovered with Ad-Hoc testing to be used as the basis for a new test case, assuming no test case already exists to cover the test path the bug was found on.


Random

A Random test approach allows the test team to execute a subset of test cases or test conditions. The subset should be representative of the whole so that the outcome will be representative of the outcome expected of the entire set.

In order to conduct Random testing there for an agreed set of test conditions and/or test cases need to be in place so a valid subset can be selected for execution.

Random testing can be of particular use where:

  • a large volume of existing test cases or test conditions could be run
  • the set can’t easily be prioritised or can’t be prioritised in any significant way
  • the organisation wishes to minimise the risk of selecting a biased subset

This is particularly common where there are very large data sets or numbers of records such as in data or record migration. In this case selecting a valid size for the subset may be done using a defined model such as that presented in ISO 2859.

As you've guessed no doubt, Ad-Hoc is not the same as Random.


Exploratory

Whenever a tester is following any testing approach and executing any type of test or test condition and then follows an unscripted or unplanned path prompted by their observations – they are following an Exploratory approach.

The Exploratory approach has at its core the idea that:

  • The skill, knowledge and experience of the tester, relating to both testing and the product, are the key to finding the most important bugs.
  • Test cases are simultaneously decided on and executed as ongoing discovery of software behaviour and learning about the software progresses.

To help focus effort a time-boxed Testing Session can be agreed along with a defined Objective or Charter for the testing. This way the tester knows how long they have and for what reason they are testing. A test lead can easily set a number of Objectives per area of the software and assign a tester to it for a time-boxed period.

The output of the testing session is of course bug reports, however there may also be written notes that can be reviewed in periodic catch-up meetings and which may form the basis for new scripted test cases.

Exploratory testing can also be run completely free-form with no Objectives defined or specific area of the software assigned for testing. This may appear to be more like an Ad-hoc approach however the major difference is in the application of the skill, experience and structured thinking of the tester.

Scripted
The most commonly used approach to delivering testing is the Scripted approach. This sees the team producing a set of test objectives and/or test cases that prescribe the testing to be conducted in very specific terms.

Fully scripted test cases are preceded by a phase of analysis in which the cases are produced. The main expectation is that test cases will not be written ‘on the fly’ during testing or that test execution will wander off the prescribed testing path.

In practice this strict adherence to just running the test cases rarely happens, except by the most junior members of the test team. As outlined in the Exploratory approach above testers will often find themselves performing a combination of Scripted and Exploratory testing at certain points whether intended or not.

Regression
Regression testing ensures that changes to existing code have not introduced bugs in code proven to work in earlier test runs. In addition a sub-set of the regression pack can be run to allow more targeted testing. An example of this would be running just the integration test cases.

When test cases have been executed and have passed they remain of use as they can be included in a Regression pack. Typically the test cases that cover the most critical functionality or the broadest set of functionality will be added to the Regression pack.

Where sets of test cases within the Regression pack are frequently used these are typically marked out as candidates for converting to automated scripts to further improve the efficiency of running them.

Confidence or Bug Finding?
Two aspects that will need to be considered when deciding on which approaches to use are:

  • The organisations focus on ‘Bug Finding’ compared to ‘Confidence’ in the status of the application
  • The type of application under test
In the diagram below we can see the approaches placed onto an analogue scale that ranges between having a focus on ensuring Confidence across to a greater focus on Bug Finding.





This is useful to help the organisation understand how the selection of approaches or combination of Approaches can support the needs they have of the testing effort.

For example, an entirely new piece of development would need Scripted testing to ensure all new functionality was covered. However this should be combined with Exploratory testing to ensure there was an appropriate focus on finding bugs.

Mark.


YouTube Channel: [WATCH, RATE, SUBSCRIBE]
or visit

Analysis techniques for user facing applications


When performing analysis as preparation to writing test cases the tester will start by using general analysis techniques such as 'Reference, Inference, Conference'.

Using these techniques, the high level explicit and implicit test requirements will be identified that mainly cover business and application level use cases and scenarios. From this point the tester must look to drill down to identify more specific test requirements and conditions.

Cases and scenarios will break down into a multitude of paths through the application and the states it can be in along those paths, paths and states will break down further into specific data driving those states, the data will break down again into specific syntax.

Use Cases and Scenarios > Paths and States > Data > Syntax


YouTube Channel: [WATCH, RATE, SUBSCRIBE]
or visit


It’s essential this decomposition of the test requirements and conditions is done so the various analysis and test case design techniques can be employed to best effect. The Development and Test teams have a large range of test case design techniques they can use, but here we'll just think about those used by the test team.

Black Box Techniques (Functional and Non-Functional)
The test team will be looking to design tests that mimic actions the user may perform, either individually or as a process. They will also want to test the way the system handles the data they use when performing these actions and the forms and structure this data could take.

Flow, State and Data
Test case design techniques can be broken down into certain types in line with what we aim to achieve by using them. They can focus on data the user may enter or the system may have stored for use, states and changes of states of the system or components of the system and the user's journey or flow through the system.

State Transition Testing (State)
The state of an application is the condition or mode that it is in. The state of the application remains static until an event occurs, at which point data is sent and a response, visible or otherwise to the user, is returned. This aspect of state transition within a system is the most critical for testing. The tester may not recognise a state change has occurred which may affect subsequent test cases.

State transition can occur for a number of reasons, including the user initiating the state change by interaction with the system, scripts being run by a service or a connected application being used.

The tester should consider:
  • what states components of the system may be in when initialised
  • transitions that can occur and what it takes to bring about that transition
  • combinations of states and how they may affect other components

Simple examples are those of a web site that uses Ajax and cookies.

- Cookies
When the web page loads for the first time it may call a script that creates a Cookie. This will be its default initial state and this state causes the local system to change. If the page is re-visited the initial state of the web page will now be different as the cookie exists. The cookie’s initial state however may change as it may be updated, if it’s expired the state of the system may change as the cookie is replaced or removed.

- Ajax
If the user then selects an option from a drop-down menu they will do so from the initial state of the drop-downs. Selecting an option in a drop-down menu changes the state of the web page and if the page uses Ajax, other elements on the page may be changed in response (visible to the user or not).


We’ll discuss next how we can use Equivalence Partitioning to reduce the huge number of possible test cases related to data. In a similar way we need to reduce the number of test cases it’s possible to identify, related to paths through the software, when we use state transition testing as a test case design technique.

As it isn't possible to test every state and path the tester needs to consider coverage and the most important states which test:

  • paths that exercise the most state-to-state transitions
  • error and error recovery states
  • the least common paths between states

Equivalence Partitioning (Data)
A major issue faced by the tester is how to minimise the number of test cases while covering as many valid test conditions as possible, especially where these cases cover functionality that can accept a large range of data.

It’s likely that where a large range of data can be entered into or used by the application under test, this will be a large range of algorithmically similar but syntactically different data. An example may be postcodes and addresses or bank account numbers and balances. Within this overall data set there will be many equivalent groups.

An example of this is where a field may take currency values between £0.01 and £100.00 (one pence and one hundred pounds). We could separate this into ranges of ‘£0.01 to £0.99’ and ‘£1.00 to £100.00’ and say any value in these two groups is essentially equivalent. Those values being either a whole pence or whole pound values, in which case we may test with data of £0.50 and £50.00. If test cases with this data pass we could assume any other case with data sitting within the Equivalence Partition would also pass.

In this event data can be partitioned into equivalent groups and a reduced number of test cases written to cover the data. This will result in more efficient testing as a valid and representative sample of the whole data set is being tested.



Boundary Value Analysis (Data)
Where data that can be entered or used by the system is within specified ranges we can say that the lowest and highest ranges sit on a boundary. Similarly where we’ve split the data range into equivalent groups the lowest and highest values in those groups also sit on boundaries.

Assessing where these boundaries sit within the data range is the purpose of Boundary Value Analysis. The outcome of doing this analysis is to identify possible test cases that may be used to find less than robust coding around these boundaries.

Examples of the types of errors that can be found using this design technique include use of incorrect logical operators or stating invalid data ranges as acceptable for entry or use.


Syntax Testing
Something that seems to get forgotten when working with data, is whether that data is syntactically correctOnce we know what type of data is allowable and we’ve used it to test ranges and boundaries are valid and have been defined and coded in a robust way, our next step is to test the data itself.

For the purposes of our discussion, data can be considered to have two attributes:
    • Type
    • Syntax

  • Data can be side to be in one of two conditions:
    • Valid
    • Invalid

Examples could be:
  • Insufficient numbers in an account field
    • valid type of data
    • invalid form of syntax
  • A string of extended characters in an address field
    • Invalid type of data
    • invalid form of syntax

Controlling the scope of Test Cases
When we use test case design techniques to identify useful test cases, we should consider the need to limit the scope of the test case.

This generally means using a test case to test for a specific condition or failure mode. For example, we wouldn’t look to test a boundary that’s likely to fail in combination with data that’s also likely to fail. The idea of ‘1 Test Case, 1 Test’ comes into play here.

In practice however it may be more efficient to have a test case that could fail in many ways. An example could be a test case that has invalid data that should be trapped, on a boundary that is out of limits. We would observe the data validation failure and once fixed, re-run the test case to fail on the out of bounds condition.

This is often how automation test cases are written in order to hit the application hard in as broader way as possible.


The merits of multiple or single test conditions, within test cases being executed manually or with automation, during the progressive or regression testing is open for discussion and as always - driven by the context of the organisation.

Mark.

YouTube Channel: [WATCH, RATE, SUBSCRIBE]
or visit




Tuesday, 11 February 2014

Ruby - Constants

The main aspect to remember about Constants in Ruby, is that their values are meant to be set when they’re initialised and then remain constant for as long as the script remains held in memory. The Ruby interpreter doesn’t enforce this however, so it is possible to change the value of a Constant. The interpreter will throw an error and it’s considered bad practice, so the advice is - don’t do it. If you are likely to want to change the value of a Constant, then consider the use of other types of variables.



YouTube Channel: [WATCH, RATE, SUBSCRIBE]

In the previous post about Ruby Global Variables, it was suggested they could be used to hold a constant value while the script was running. However, in our example we used a Global variable to leave open the option that the opening Hero Name might be assigned by the player. There’s nothing stopping you using a variable of one type or another and keeping the value constant during the life of your programme or going ahead and using a Constant. Just make sure you set it up properly and that others reading your code will know your intentions.

Type the following into a new .rb file and run it via a CMD window
#Test of assigning and then reassigning a value to a Constant

FIRST_CONST = 10
SECOND_CONST = 20

puts "Value of the First Constant is: #{FIRST_CONST}"
puts "Value of the Second Constant is: #{SECOND_CONST}"

puts "Now we'll change the value of the First Constant"

FIRST_CONST = 30
# We should see a warning message now, but the value will be changed
puts "Value of the First Constant has been changed: #{FIRST_CONST}"

If you try to use an uninitialized Constant, you’ll get a NameError exception thrown.
Type the  following into a new .rb and run it via a CMD window.

# Testing for the NameError exception on an unititilised Constant
puts "Here's what happens if the Constant hasn't been initialised"
puts "#{DOES_NOT_EXIST}"


Also, try the following to show that a Constant doesn’t exist if it has no value assigned.

# Showing a Constant doesn't exist if it's had no value assigned
EMPTY_CONST
puts "Does a Constant with no value exist?"




Naming Constants
Constants must begin with an upper case letter, for example Inventory. From the previous post on Instance and Class variable, you’ll perhaps realise via this naming convention, that every Class and Module name creates a Constant. However, they are usually written using Camel Case which helps differentiate them from Constants, e.g. InventoryCheck.

To help keep a differentiation between Class and Constant names there are two common conventions for their naming:
  • Using all uppercase letters, e.g. INVENTOR
  • Combining with underscores if needed, eg. MY_INVENTORY



Scope of Constants
As with our Local, Global, Instance and Class variables, there are some scope rules to bear in mind when using Constants within your Ruby scripts:
·         Constants can’t be defined in Methods (dynamic assignment error, see below)
·         Constants defined in the main body of the script can be accessed globally
·         Constants defined within a Class or Module can only be accessed from within those Classes or Modules

Try the following in a new .rb file and see what happens with accessing a Constant within a Class

# accessingConstants_001.rb
# This re-uses the Class called Creature that we saw in earlier posts

# We'll create a Constant outside any Class, Method, etc. to see if it's Globally accessible
MOB_INVENTORY = "The creature's Inventory is empty right now.\n"

class Creature
  @@mobType = "NPC"
  MOB_WEAPON = "The Mob's weapon is a Club"

        def initialize creatureName
            @name = creatureName
          end #def initialize creatureName

        def to_s
            "Creature name is #{@name}. It is an #{@@mobType}.\n"
          end # def to_s

        def inventory
         puts "Called from within the Class: #{MOB_INVENTORY}"
        end

        def mob_weapon
         puts "Called from & defined within the Class: #{MOB_WEAPON}"
        end   

end #Creature class

# Create a new instance of the Creature class, then assign it a name via the @name Instance variable
mob_001 = Creature.new "Silver Kobold"

# Access the Constant directly, as it's declared in the body of the script
puts "Called from main body: #{MOB_INVENTORY}"

# Access the MOB_INVENTORY Constant via the CLASS, showing it has Global scope as the Class can reach it
mob_001.inventory

# Access the MOB_WEAPON Constant defined within the CLASS itself
mob_001.mob_weapon




As described in the comments we can see three things in action:
  • Use of the Constant MOB_INVENTORY via a Class, showing its Global scope
  • Access the Constant MOB_INVENTORY directly outside any Class or Module
  • Access the MOB_WEAPON Constant, defined in a Class, via the Class


For a final test, try the following code in a new .rb file

#constantDefinedInMethod.rb

def mobSpell
        MOB_SPELL = puts "A Constant defined in a Method"
  end

As we’ll see, this fails with a ‘dynamic constant assignment’ error. The issue here is instead of assigning a static value, we’re trying to dynamically assign it each time the Method is used.



That concludes our overview of Ruby variables; Local, Global, Instance and Class, along with our look at Constants, which we might also find use for when creating our scripts.

Next up, we’ll look at Ruby Arrays, a great way to store and work with sets of data. Taking us beyond the variable and its single piece of data.

Mark.


YouTube Channel: [WATCH, RATE, SUBSCRIBE]
or


Read More
http://cyreath.blogspot.co.uk/2014/02/ruby-instance-and-class-variables.html
http://cyreath.blogspot.co.uk/2014/02/ruby-global-variables.html
http://cyreath.blogspot.co.uk/2014/02/ruby-local-variables.html
http://cyreath.blogspot.co.uk/2014/02/ruby-variables-categories-and-scope.html
http://cyreath.blogspot.co.uk/2014/01/ruby-variables-part-1.html
http://cyreath.blogspot.co.uk/2014/01/ruby-getting-and-using-user-input.html
http://cyreath.blogspot.co.uk/2014/01/download-and-install-ruby.html



Ruby - Instance and Class Variables

When we have an instance of any kind of object in Ruby, it will have certain variables that are unique to it. These variables are therefore referred to as Instance variables and they are denoted by the use of the @ sigil. You’ve probably read that in Ruby everything is an object, therefore we’re practically always going to have a collection of instance variables available to call on.



YouTube Channel: [WATCH, RATE, SUBSCRIBE]

Class variables are set-up when we define a Class in our script, they are denoted by the use of the @@ sigil and are common across all instances of the Class.

It’s important to note that we define our Class variables when write up the Class, with the structure we want, at the time we create the script. These will be re-used by all future instances of the Class and will carry the same data into all instances.

The value of individual Instance variables is set when we create an instance of the Class when the script is running. The data / value it holds is unique to the instance and gives it some variation on other Instances.

Let’s look at an example which shows @instance and @@class variables being used.

Copy the following code into a new .rb file, then run it via a cmd window.

# instanceAndClassVariables_001.rb
# Define a Class called Creature and set a Class variable with @@
class Creature
  @@mobType = "NPC"

def initialize  creatureName
    @name = creatureName
  end #def initialize creatureName

  def to_s
    "Creature name is #{@name}. It is an #{@@mobType}."
  end # def to_s
 
end #Creature class

# Create a new instance of the Creature class, then assign it a name via the @name Instance variable
mob_001 = Creature.new "Silver Kobold"

# View the details of the Class, calling the Class and Instance variables
print mob_001



In this example of a simple Class, we see both Instance and Class variables being used. Our Class variable is @@mobType and this example it’s an “NPC” (a Non-Player Character). We can tell that all instances of this Creature class will be the NPC mob type. However, the particular name of this instance of the creature isn’t set until we create an instance of it.

YouTube Channel: [WATCH, RATE, SUBSCRIBE]
or

Friday, 7 February 2014

Guidelines on conducting a Face to Face Interview

When conducting face to face interviews, it's good to have a format for them. A format that is consistent, fair on all candidates and allows you to evaluate each person on a like for like basis. Scroll down for links to some other useful items to help in the interview process.


Face to Face Interview Outline


1)   Greet the candidate
Welcome the candidate to the interview and introduce yourself and what your role is in the business and the interview.

2)   Confirm information
Show the candidate the CV you have of them and any Job Specification they should have seen. Make sure you and the candidate have the same information.

3)   Outline the interview Process
Explain how long the interview will run for, that there will be a brief Candidate Assessment and that the main focus will be on talking through the candidates CV, exploring their experience and knowledge in context of the role on offer, business needs and their own aspirations.

4)   Begin the Interview
Conduct the first part of the interview for around 30 minutes.

5)   Break for the assessment
Leave the candidate with the Analysis Assessment for around 30 minutes.
(see link below)

6)   Close the interview and thank the candidate
Advise the candidate that we’ll review the interview and advise them within the next few days, set a date if possible.


Required Items:


  • Candidate CV
  • Job Spec if one has been set
  • Management Org Chart
  • Copy of the Assessment
Straight forward stuff, make sure you and your colleagues adhere to this or some other consistent approach and interviews will go a little smoother.

Have a look over the previous blog post for what can go in the Assessment.


You might also find the following templates useful too:


Mark.