Smart Contracts

Updating Solidity code and Testing a Smart Contract

Books on the Blockchain

Publica Self Publishing

Goodbye Contracting

Hello brave new old world...

Ruby-Selenium Webdriver

In under 10 Minutes

%w or %W? Secrets revealed!

Delimited Input discussed in depth.

Thursday, 14 August 2014

Why you're NOT a Software Testing Expert, probably

I see it often on people's profiles and CVs, Expert. It's one of those words that get my attention straight away - and is often badly defined. I'm sure that trying to work out where you are, on a scale from clueless to expert, is something I'm not alone in doing. I was recently invited to review a training course on the basis feedback from an expert like me would be useful. Hmm...

In a recent set of interviews for my next contract and (shock) a permanent role, I've been thinking about this more than often, as you would. It's important to be clear about what you do and don't know, then answer truthfully... no really ;). In some of the calls I've been asked to rate myself on a scale of 1 to 10 either for specific skills and technologies or say as a tester or manager. This creates a problem, given my expectations of what an Expert is, being set so high. The problem being, I rank myself much lower than perhaps I would be expected to say. I could state big numbers, but I don't have an ego the size of a planet. I probably don't have 'marketing' smarts either.

Thinking through what an expert or a 10 is, I came to the conclusion that people at this level are almost unheard of, certainly rarely encountered and almost definitely not doing telephone interviews. That's why you're probably not an expert, probably. We could argue on the 'label' but let's use Expert as the top and Clueless as the bottom. I'll make up ones in between. The reason you're probably not an expert is due to the second issue, the labels are used too broadly.

Imagine I introduced myself as an Engineering Expert, Medical Expert, Physics Expert, Agricultural Expert, Legal Expert. You would immediately need to ask "In what field?" or maybe "In what area?". These descriptions are too broad. There's no way you're a 10 in brain surgery and a 10 in gerontology, you won't be a 10 in family law and a 10 in contract law, you can't be a 10 in civil engineering and a 10 in aeronautical engineering. You can't therefore be a 10, a Software Testing Expert. By definition it doesn't make sense.

Just at a high level we could break testing down into say Functional, Performance and Security, though we could add in other major testing areas. Straight away with just these, I'm sure you'll agree we all know people who are more 'expert' in some of these areas than others. Add in that there are people who focus on test management or on the other end of the scale perhaps automation. Even then, when we say "they're an automation expert", we mean they have expertise in a certain set of tools, technologies and techniques (TTT).

OK then, so when asked to rate ourselves and if we're an expert how might we define that?

On a scale of 1 to 10...

1 - Clueless
You have no awareness of the TTT. When someone says Git, Capybara, C and Java to you, you think it's about animals who are idiots living across the sea on an exotic island.
But hey, well done for reading this blog, stay a while, click an ad. Now go and get yourself the ISTQB Foundation certificate and become a real expert! (that was a joke...).

You resolve to continue pushing yourself. Perhaps there's more to you than you thought.

Given the above, the rest of us fit somewhere after. However, one more refinement, we don't need to worry about exact numbers. In any given role your skills and aspects of them, will be needed to a greater or lesser degree. "about a nor n, depending on the demands of the project" is a reasonable answer when asked about your skill, on a scale of 1 to 10.


2 to 3 - Beginner
At this level you've been exposed to the technology, though your understanding of it is limited. You are starting to remember what you are shown step-by-step to do, though constant guidance is needed when encountering issues outside of direct tuition. As a beginner you won't yet understand how the TTTs fits into the overall testing domain. You may recognise the need, e.g. why continuous integration is a good idea, why end of day reporting is valuable, but you carry out tasks as shown, in a rote fashion. With more practice, you start to take on more complete tasks under guidance and begin to need less day to day reference to mentors or help sources.

You realize that all your life you have been coasting along as if you were in a dream. Suddenly, facing the trials of the recent past, you have come alive.

4 to 5 - Able
Now you've started to gain some ability to use the TTT day to day as expected. You're fully comfortable with the basics and have started to understand why the TTT is needed. In addition, you now start to see what value it adds, how it integrates with and supports other TTT. You can be readily delegated to but might not always be completely successful or fully efficient in using the TTT to complete a task. Some remediation by more experienced staff might be needed, though when given you gain useful insight into whatever was previously unclear and make positive advances in your overall ability.

You sense yourself more aware, more open to new ideas. You've learned a lot about Software Testing. It's hard to believe how ignorant you were -- but now you have so much more to learn.

6 to 7 - Capable
At this level you're able to carry out whatever needs to be done, with guidance and support not expected to be needed. You fully understand how the TTT works and integrates with others, have insight into common and more obscure issues and problems that occur. You can now start to mentor others in the team and comfortably bring the TTT to projects as part of your professional tool-kit. When thinking of what you 'do' and how you work, the TTT will be part of how you define yourself and your capabilities. many sources of information on the TTT and others people's understanding of them are notably less than yours. You may start to find it difficult to discover new ways of improving your understanding and use of the TTT because what's to learn is getting less and less mainstream, more difficult to find.

It's all suddenly obvious to you. You just have to concentrate. All the energy and time you've wasted -- it's a sin. But without the experience you've gained, taking risks, taking responsibility for failure, how could you have understood?

8 to 9 - Competent
Once you become competent you've gained a full professional understanding of whatever TTT you're working with. By definition you are now possessing all the necessary ability, knowledge, experience and skill to do something successfully, efficiently and with complete consistency. Doing it with your eyes closed... on auto pilot... hardly thinking about it - are all phrases that people are whispering behind your back. You will certainly understand the shortcomings and limitations too, identifying ways to work around them or improve them and integrate with other TTT. By now you'll be known as a key go-to person for the TTT in the profession and likely blog or write on the topic. You competency is so complete, that you can begin to subvert the methods and reasons for using the TTT to deliver unexpected improvements and start to re-wire the ways of working and benefits derived.

You're really good. Maybe the best. And that's why it's so hard to get better. But you just keep trying, because that's the way you are.

10 - Expert
There has to be a level that is the absolute pinnacle of knowledge and expertise, a level so high it cannot be surpassed. This is the level 10 Expert.

To be a level 10 Expert you invented the TTT, several years ago and others are still playing catch-up. You published a book on it and/or get speaking engagements to explain the awesomeness to the masses. Alternatively, you have gained such a level of understanding that you have gone back to the inventors and pointed out where they're getting it wrong, heck you probably decided they're so missing the point that you went and spun off your own TTT to show how it REALLY should be done. You don't look for jobs with this TTT, they come to you.

You'll never be better than you are today. If you are lucky, by superhuman effort, you can avoid slipping backwards for a while. But sooner or later, you're going to lose a step, or drop a beat, or miss a detail -- and you'll be gone forever.

(it was going to be this, but the above was wittier ;} )

Like a Spartan of the tech world, you search the globe for your equal, but it's a lonely pilgrimage - Mark Crowther

Extra bonus points if you can place the quotations...



Liked this post?


Tuesday, 5 August 2014

Ruby Basics - Wrap Up 1

Hey All!

Well here we are, 15 videos in and already at a wrap up of what we've learned so far!

In this post, let's do a code walk-through of a script that includes the elements we've covered in the rough order of the videos. Be sure to check out the Ruby Playlist on YouTube if you've not done so already. Grab a copy of the script here


The first thing to look at is getting user input and assigning the data to local variables. Here we do a basic puts and using a local variable called userFirstName we assign it a value the user enters. On this value we call the string class methods of chompdowncase and capitalize. In the second set of lines, we use #{interpolation} and call a string method on that too. Quite a lot in 4 lines!

# Let's declare our LOCAL variables and get some values straight away (Video 2 and 5)
puts "Welcome, what's your first name? (Dave, Alan or yours)"
userFirstName = gets.chomp.downcase

puts "Hi, #{userFirstName.capitalize}. What's your surname?" 
userLastName = gets.chomp.capitalize

Next we assign a global variable so we have access to it anywhere in our script, pretty similar to what we've just done. Global variables are identified by the $ symbol.

# Let's create an example GLOBAL variable (Video 6)
$globalNameText = "User name is: "

While we're looking at variables, let's declare an instance variable and so some concatenation. You'll recall that instance variables are identified by using the @ symbol. (We'll look at Class @@ later). We also snuck in another string method, upcase.

Here there are two ways to concatenate text, using either the + symbol or << symbol. Ruby docs tell us that + creates a new string, where as << appends the string to whatever precedes it. In terms of speed and memory usage this could be significant on large data sets! Let's be clear, userFirstName + " " creates a new string that now includes a space. Doing << UserLastName just adds it to the existing string.

Can you design a test to prove this?

# Now we'll make an INSTANCE variable from the user name (Video 7)
@userFullname = userFirstName.upcase + " " << userLastName
#upcase is also string method 

Next we define a Constant, which we do by using all uppercase. Age isn't a good exmaple but it works for our purposes :) As we're getting a number, we don't use the usual downcase, as that wouldn't make sense. Try it and see what happens.

# Finally let's get a CONSTANT in use (Video 8)
puts "#{userFirstName.capitalize}, how old are you?"
USER_AGE = gets.chomp #we UPPERCASE Constants to differentiate against variables

We print out a message using our Global and Instance variables, showing concatenation works just fine on these too.

# Put the correctly formatted name to screen
puts $globalNameText + @userFullname

As we want to have data already stored for our program to use, we next create a simple array of data, just with strings in this example.

# set up an array with the template roles (Video 9)
rolesArray = ["Developer", "Tester"]

Now that we have our data, let's do some evaluation of it and decide what the outcome of that should be. The first step is a Case statement, against whatever names the user entered above.

Here, we respond to the user with data we've pulled out of the array we just set up. You'll recall that we index into arrays starting at position 0. So when we run the script, Dave will be assigned the role at position 0, a Developer.

# Depending on what the users first name is we'll respond with more details (Video 10)
case userFirstName
when "dave"
    puts "#{userFirstName.capitalize} you are a #{USER_AGE} year old #{rolesArray[0]}"
    userRole = rolesArray[0]
  when "alan"
    puts "#{userFirstName.capitalize} you are a #{USER_AGE} year old #{rolesArray[1]}"
    userRole = rolesArray[1]
    puts "You must be a new member of staff, welcome!"

Next, we'll ask about a career change and instead of using a Case statement to control the flow of our response, we'll use a basic if and then a nested if statement.

As we've discussed before, we can evaluate the response using regular expressions or direct evaluation.

# Here we use a nested IF to check for career changes (Video 12)
puts "Do you want a change of career? (Yes or Y or No or N)"
careerChange = gets.chomp.downcase

if careerChange =~ /\A(yes|y)\z/ then
    if userFirstName == "dave" # (Video 11)
      puts "#{userFirstName.capitalize}, you are a now a #{rolesArray[1]}"
      userRole = rolesArray[1]
      elsif userFirstName == "alan" or userFirstName == "Alan"
        puts "#{userFirstName.capitalize}, you are now a #{rolesArray[0]}"
        userRole = rolesArray[0]
        puts "Easy #{userFirstName.capitalize}, you just joined us!"
  elsif careerChange =~ /\A(no|n)\z/ then
    puts "Great, keep up the good work!"
To finish off the main body of the example script we'll now use a While statement, to keep asking a question until a condition is met. We've also snuck in an If ternary to decide how to respond if the hoursWorked value is under 8.

# Now we'll check if the user has done a days work! 
hoursWorked = 0
userRecord = [userFirstName, userLastName, userRole, USER_AGE]

while hoursWorked < 8 # we could do an UNTIL hoursWorked == 8 here instead   # (Video 13)
  puts "How many more hours have you now worked #{userFirstName}? (enter 0 to 8)"
  puts "total hrs worked is so far: #{hoursWorked}"
  hoursWorked = hoursWorked + gets.chomp.to_i

    hoursWorked < 8 ? (puts "Keep going, the day's not over!") : (puts "Well done, go home and relax.") #example of if Ternary (Video 13)

Just to finish, let's add a new data element to the end of the UserRecord array and print the results.

userRecord.push hoursWorked #Pushing to an array (Video 14)
puts userRecord

Don't forget to grab a copy of the script and play through it yourself!


Read More

Monday, 4 August 2014

Negative and Positive Tests - you're getting it wrong

When we write test case sets, there's a commonly used idea of writing positive and negative test cases. The thinking is that positive test cases cover all the things you DO want the software to do, while the negative test cases cover what you DON'T want it to do. However, when applied in this way the negative test cases usually get confused in their focus.

All features and functionality that we code in intentionally, are POSITIVE scenarios. Yes, any error conditions, retries, warning dialogue boxes, etc. that are specified as requirements or standard expectations are POSITIVE test conditions. When you test the failure path, with expected outcomes, it's a Positive test case. Negative test cases need to cover the unintended and unexpected. If you thought writing Negative test cases was easy, you've definitely been doing it wrong.

So what do Positive and Negative really cover? In my view it's the following:

Intended, Expected and explicitly stated features and functionality
- that are required - the stuff you know you do want
- that are not required or expected - the stuff you know you don't want

Potential issues and problems that can occur even if the probability is low;
- that experience tells you are possible given the technology you're using (inductive)
- that you believe are likely given the nature of issues you've seen before (deductive)

Writing Positive test cases^ is relatively easy, writing negative test cases is hard, period. To do it well, you have to draw on a combination of technical knowledge, systems know-how and raw experience. A way to do this is to apply Inductive and Deductive analysis. I've written about that in a separate post, but here's a recap:

- Inductive: Define a failure mode / bug and think of how you would know it was present in the system, what would failure look like?
- Deductive: Define some form of erroneous behaviour and work out what would be broken in the system for that behaviour to occur.

With this approach you come from one end or the other to the same conclusion, defining a failure mode and it's effect. Yep, if that rings a bell it's because it's not some super new way of thinking, it's been around since the 50s in the shape of Failure Modes & effects Analysis (FMEA). It's a key ingredient in the sauciness of being a test architect.

As a rule of thumb, expect to write a meaningful Negative test case for every 50 Positive test cases, that ratio wouldn't be unreasonable at 1 : 100. These are the tests that stop planes falling out of the sky, prevent satellites from failing and stop nuclear power plants exploding, so don't stress it if your numbers are low.

Pass or Fail?
When writing Positive and Negative test cases (or unit tests for that matter), a common question is whether they should be shown as Passes or Fails. Positive test cases should certainly pass, we're testing for something that we know should or shouldn't happen after all. When that expected behaviour does or doesn't happen, it's a pass. but for Negative? I see the argument for flagging these as Fails. They should Fail, a Fail of a Negative is a good thing, it means planes are in the sky and the world it's Fukashima'd.

However, in practice this approach proves confusing. The discussions with management about ongoing status and end of test reporting, where Negatives are passing when they're showing as bright red fails is just too much work. Pass them all. When a Positive passes it's green, when a negative fails it's green. When a negative passes however, you have a major unexpected issue and need to jump on it.. Separate out your Positive and Negative test reporting for extra impact and kudos.

^ Test Cases, Mind Maps or whatever your test currency is for your project.

Common Examples
It's hard to come up with a cookbook of Negative test cases ( I may have mentioned that...), but here's some I get reuse out of from time to time.

- Configuration files missing
- Entry to the system without authentication
- Web server not operational
- User able to access other user records
- Data or file encryption not working
- Backwards compatibility failure when upgrading to a new version of

These are common across many systems, but are examples of 'that could never happen'. We hear this too often as tech stacks usually come with whatever it is already included or enabled, it's core functionality beyond that being worked on day to day. Therefore it 'would just never fail', 'no one would make that change without checking' or the system 'couldn't work without it'. Hubris before the fall. The risk is there and putting tests in place to make sure the failure mode is caught as soon as it occurs is critical.

With a slightly shifted perspective on Positive and Negative tests, the robustness of your test regime can be increased greatly. It takes practice to write them well, but your application needs them!