Excel as a Test Management Tool

Because you're going to use it anyway...

Ruby-Selenium Webdriver

In under 10 Minutes

%w or %W? Secrets revealed!

Delimited Input discussed in depth.

Managing Knowledge Transfer Sessions

Read the article, grab the templates

Ask questions

If you don't ask, you won't learn. You won't teach either.

Sunday, 29 March 2015

Understand the context of the Testing Problem

In various other posts and papers, I've said that testing is testing. That if we’re suitably skilled and experienced in our profession, then we can test in most any environment and industry. I believe this remains as true as when I first said it.

To quote directly from the FX Kickstart for Testers paper;

As a competent test professional you’ll know that fundamentally you can test in any domain. That domain could be travel, healthcare, engineering, retail, etc. and of course finance. Ultimately we can say with sincerity, it’s all software testing and the skills and knowledge of our profession that we possess, apply no matter what the domain.

However, there are two very real caveats with this perspective that will limit our effectiveness in a given domain until overcome:

  • We will initially lack understanding of the testing challenges in the domain, thereby limiting our application of contextually relevant testing techniques and approaches
  • The domain will have unique vocabulary and concepts that must be understood if we are to translate its meaning and communicate effectively
Once we address these caveats we will understand the unique testing needs of the domain in a contextually relevant way and be able to communicate with our colleagues effectively.

The point is, to deliver effective and pragmatic testing we need to understand the client’s business. Let’s look more closely at the caveats above.

A lack understanding of the testing challenges in the domain
A mistake inexperienced consultants make, is to roll into a client’s site with a pre-concieved idea of what a testing solution will look like, yet they have scant knowledge of the problems the client are facing.

We see this with many of the larger consultancies. Their team arrive with all the answers already cooked up. The direction a solution will move in, the shape it will take, tools to be used, etc. Are all mostly known in advance. That’s not always a bad thing. If it’s known the testing problem is  the integration of new development with existing systems, then certain approaches for this will be known. Maybe the problem is specific, a lack of security testing for example. In which case there are some industry standard, yes even best practice, approaches to take.

On a new engagement it’s important to take the time to put off solutionising and ensure we’re clear on such things as:
  • The nature of the problem is known
  • What problems are being caused to current business operations, what limitations it is causing
  • What is limiting the client from solving the problem themselves
  • What attempts have been made to solve the problem and what the outcome was
  • Timescales for the problem needing solving and why, is anything dependent on the test work

Unique Vocabulary and Concepts
I stated above this applied to our ability to understand the language of the business to ensure clear communication, in a dialect the business would understand. It’s more than that however. Just saying an environment is System Integration instead of Development Integration or the other way around is not the big issue. Though it certainly helps to get that right too.

The big issue is with the concepts and I would expend that to be quite inclusive; the ideas, ways of thinking, ingrained culture, internal politics, external pressures, etc. Understanding these are just as important to the success of your consulting engagement as knowing what the testing problem actually is and how they talk-testing in the business.

We should check if we’re in need to understand other aspects such as:
  • What kind of environment the client has in place culturally; open, communicative, blaming
  • How they work technically; adaptive / descriptive, predictive / prescriptive, hybrid
  • If they are subject to professional or legal compliance and auditing
  • Any issues with public perception or on a branch / regional basis

Have a look at the free preview pages for Flawless Consulting, scroll down to where it says 'Consulting Skills Preview'. This is valuable additional reading.

Testing is just like – software testing. We can be confident in that.

What changes is the setting in which the testing problem we need to solve exists. To ensure our testing is relevant and effective, we serve the client best when we understand the context of their testing problem.

Enjoyed this post?
Say thanks by sharing, clicking an advert or checking out the links above!
Costs you nothing, means a lot.

Replacing QA with 100% automation

Replacing QA with 100% automation

A question came up on one of the forums recently, that echoes a question I and I’m sure you’ve been asked at some point. That question being along the lines of reaching 100% automation. Groan… yes, that old chestnut.

As experienced testers we already know the answer, for the avoidance of doubt the answer is: No.

Thanks for reading, now on with my day!


OK, OK… we wish the ill-informed question could be dismissed so easily.

However, though we know its asked by the ill-informed, we can’t blame people for asking. To senior management, non-technical, non-testing staff it’s a reasonable question. It’s our job to respond in a way that ensures we leave them informed.

To whit… if asked, here’s a few thoughts on why we can’t achieve 100% coverage via automated testing. (We’ll skip the QA Vs Testing perspective for now, hey they’re ill-informed, just roll with it).

You could also send them this link of Case Studies on Automation, by Dorothy Graham (RTFM…? :)

It isn’t possible to achieve 100% test coverage of the application/system with manual testing, so strictly it can’t be possible to do the same with automation. Though interestingly we might expand coverage…

Technical Limitations
There are aspects of the system you can check and test manually, that can’t be automated ‘technically’. Examples include Accessibility and Usabilty, others include complex end-to-end scenarios that might require exploratory or dynamic testing dependent on variable results.

Automation of what?
The different levels of testing (unit, integration, system, etc.) mean we’d need to have suits of different automation tests, focusing on different aspects of the system / application.

Tools and Team Skills
If we wanted to ‘automate everything’ we’d need a host of tools; web front end, application interface, API testing, webserver/DB/CDN testing. Maybe 100% automation is just one of these?

Assuming we define what 100% means in this context, do we even have the tools in place or the staff who know how to use them? If not, there will be some manual testing going on.

Don’t have time / money
The Return on Investment from automating everything just won’t be there. So, even if we had the tools, time, skills and a limited automation remit – it’s highly unlikely the time and money needed would be worth it. Automating all of a payment system that’ll be there for 10 years is more likely than automating a website that’ll be trashed in 6 months.

What other reasons can you think of?


Enjoyed this post?
Say thanks by sharing, clicking an advert or checking out the links above!
Costs you nothing, means a lot.

Site Update - Testing and Technology Library

Just a quick update, as it would be easy to miss...

The eagle eyed among you might have noticed I've redesigned the Books page of the main website, with a similar widget appearing on the right of the Blog.

Instead of a few books I've reviewed and making it not worth visiting, the Books section is now a Library of Testing books with a smattering of Technology that’ll help with those testing jobs you get landed with.

What does it contain?
The Library is a collection of books that either a) I’ve read over the years or b) come with strong recommendations. As such, it’s not just a bucket full of books, you should recognize them. For example, check out the book by Lisa on Agile testing or the one by William Perry – recognize those?

Good, all items in the Library should be ones that have proven themselves in our profession. That applies to both the books and the hardware, have you got one of these or one of these on your desk? I rest my case then ;)
  • Do you have a favourite book you want included here?
  • Do you have anything good or bad to say about the books in my Library?

Leave a comment and let me know!


P.S. Challenge - Join Goodreads and set yourself a 2015 Reading Challenge

Enjoyed this post?
Say thanks by sharing, clicking an advert or checking out the links above!
Costs you nothing, means a lot.

Saturday, 28 March 2015

Testing Books - 4 of the classics

There are a few 'classic' books about software testing that should be on every serious tester's shelf. Though written a few years ago, they still contain timely advice that's as applicable now as it was when first published.

Agile testing
I would suggest that if you read nothing else, you should be reading through Agile Testing: A Practical Guide for Testers and Agile Teams, by Lisa Crispin and Janet Gregory.

Since its first publication the book has had rave reviews and Janet and Lisa remain today as leading lights in defining what agile testing is and how it can be applied to projects.

As the blurb for the book states, readers will come away from this book understanding
  • How to get testers engaged in agile development
  • Where testers and QA managers fit on an agile team
  • What to look for when hiring an agile tester
  • How to transition from a traditional cycle to agile development
  • How to complete testing activities in short iterations
  • How to use tests to successfully guide development
  • How to overcome barriers to test automation
The book is not just a wall of text, but provides a number of diagrams and mind-maps you’ll keep referring to time and again. The authors draw on their experience greatly, sharing examples of situations they’ve encountered and what they did to overcome them.

Grab a copy of Agile Testing: A Practical Guide for Testers and Agile Teams

Lessons Learned
If you're aware of the work of Michael Bolton, James Bach and Cem Kaner around contextual and rapid software testing, then you should already have read the book Lessons Learned in Software Testing. Written by Cem Kaner, James Bach and Bret Pettichord you're guaranteed it's packed full of useful information and ideas.

Broken down into 293 lessons, the book allows you to dip in and read just what's of interest at that time. Think of this as a collection of related essays, where you can read around one topic or read across topics to get a fuller sense of a particular area.

Whether you're a veteran tester or just starting out, this is a great reference book to have on your shelf.
Get your copy of Lessons Learned in Software Testing

Test Design
It feels like this book by Lee Copeland has been around forever. When it was published, it pretty much became the oracle source on test design. A Practitioner's Guide to Software Test Design remains one of the most comprehensive books about test design, a critical skill that we need to apply almost every day.

While the book does focus on the IEEE 829 view of the world, this is no bad thing. Copeland uses that as a framework around which to provide a background to the testing context and then move onto design techniques and their effects in a way that is extremely applicable.

If you ever wanted to really apply Equivalence Class Partitioning, Boundary Value Analysis, Domain Analysis, Use Case Testing and other powerful techniques, this is the book for you.

Upgrade your test design skills by getting a copy of  A Practitioner's Guide to Software Test Design today.

Systematic Software Testing
I don't believe this book is as well known as it should be, like Copeland's book, this is a a heavyweight gem to be sat on all tester's shelves. Unlike Copeland's book that focuses on test design for execution, Systematic Software Testing focuses on the process and practices fundamentals of software testing.

The book focuses on process and practive, makes reference to the IEEE standards but then applies years of experience and know how to subvert that process and show you how to really build a testing practice that'll work.

There's information on the information you'll need, people and documents. The author relates the testing process and practice to the wider development context and ensures the reader is set to deliver well technically as well as a member of the business.

Another must read, so get Systematic Software Testing and round off your understanding.


Enjoyed this post?
Say thanks by sharing, clicking an advert or checking out the links above!
Costs you nothing, means a lot.

Excel - Best. Test. Management. Tool. Ever.

OK, I may be talking that one up, Excel is not the “best test management tool ever”, but I bet you either of the following ring true:
  •          It’s as used a most of the big name test management tools
  •          It’s what gets rolled out when there is no tool in place
  •          You just did some test/testing management stuff with it recently
There are a number of good quality, free and low cost test management tools out there, but even with that we’ll end up using whatever;
  •          The customer already has in place, paid/free/other
  •          Is easiest for the test team and stakeholder community to work with
I’ll bet you again that where a tool isn’t in place that, whatever-test-management-tool-is-easiest-to-work-with, is Excel.

Given this, we need to make sure we’re up to speed on how to make useful Excel documents that can provide, capture and report testing information and status.

Let me emphasise this again, you ARE going to use Excel extensively in your testing career – so invest some time learning it! Now, this may use may not be so prevalent in a more agilistic environment, but don’t bank it. Just because “we’re agile here” doesn’t mean the definition of what tests will be run and what the outcome was suddenly goes away.

Use it for what?
What might we used Excel for?

  • Test Cases – defining them and capturing test run information
  •  Coverage Matrix – defining system coverage by test cases and tracking status of coverage
  • Test Status Tracker – a dashboard of status for test preparation and execution
  • Status Dashboard – analyse data, conditions, status, etc. and present a view to the world
  • Burn Down Dashboard – Number of test cases over time against actual? Excel will map this too
The above might be separate Excel workbooks or combined, they might be cross linked or standalone. You decide what’s best.

As always, there’s no hard and fast rule of what you’ll need to use or how you’ll use it. Take the templates and ideas as just that, then build them out to meet the specific needs of your software testing project.

But why EXCEL?
Most of what we deal with is the presentation, capturing and representation of data with some guidance narrative text. That its best in Excel. Sure, you could have Test Cases in Word for example. But how are you going to analyse the numbers and conditions of the data sets? How will you create pretty charts and dashboards?

Just stick with Excel and use other tools for the management extracts you might be called on to provide for the senior levels in the business.

Using Excel is not without issues. The biggest problem is you’ll most likely en up emailing the document around. Uh oh… filling everyone’s email box up and worst of all - version control nightmare. My advice is don’t do it. Stick it on a shared drive or better still use some extra tool like SharePoint, assuming the client has this and they’re a Microsoft house. You can at least check-out the document then so people know you’re editing it.

Middle ground is have the document on a shared network drive and use the ‘Share Workbook’ feature. You might what to practice using this before doing so in anger.

Be VERY careful clicking that orange highlighted check box...

At worse, stick with an agreed document versioning method and sort out How You Do Documents™. Have a look at my paper on that very topic: A Documentation Model

Where to Get Excel
So, you’re sat here starry eyed at the thought of all the amazing Excel in your future, but what’s that, you don’t have access to a copy?! I would be stunned if this was the case these days. Apple computers haven’t taken over like Windows and office is usually there somewhere on a Windows machine.

If not… then hit www.live.com or possibly www.outlook.com and sign up for an account. With that you’ll have online access to the office suite and email.

If you need a personal install, go grab a copy from Amazon, using my affiliate link of course (you get extra karma points for that).

Be sure to have a look over the site and review the various Excel templates available there:

Have fun!


Enjoyed this post?
Say thanks by sharing, clicking an advert or checking out the links above!
Costs you nothing, means a lot.

Friday, 27 March 2015

Keep what you’ve got, by giving it all away

I remember working at an electronics manufacturer many years ago. The managing director there was an interesting guy, interesting in that his attitude was a little bit back in the 1970s.

One conversation I had with him was about training. My argument was that we should implement a simple training course, more a skills check, for the staff on the line. The idea being that we could ensure a consistency of approach and spot any training needs. After all, getting staff into the business, a new start up at that time, then trained on the products, was a big investment. It was surely wise to ensure everyone was around the same skill level for the core skills.

His response was straight out of the How Not to Manage handbook. Yep, “If we train them, they could leave.” As the retort goes, “… and if we don’t train them and they stay?”
This attitude has no place in the modern professional environment. It’s a small minded and fearful attitude. One that reeks of ‘lack’ instead of ‘abundance’. It’s an attitude that if applied to other matters, will help kill off any team, business and even personal life. When consulting or working for an employer in the software testing field, this attitude will be your death knell.

Small Minded Consultants
I recently encountered a similar attitude to this again, in my current testing consultancy role.

The scenario here was a conversation with another consultant. I proposed that we should collaborate on some White Papers, proposing service offerings our client could deliver. Showing them how to structure the services, tools needed, likely costs, blockers and enablers. All you’d expect in something that was essentially a business proposal.

The response from the consultant, 15+ years on since the encounter at the electronics company, was like hearing an echo. This time the complaint was that if we show them what they could do in such detail, they wouldn’t need us, they would go off and do it themselves.

Why this is wrong, wrong
This is an attitude of fear, of small mindedness and a reflection that this person’s consultative mindset is way off.

First off, I absolutely believe you cannot keep secret everything you know, in an attempt to keep it rarified, believing it will keep you employable. In doing so you, if it’s something new for your client, you will fail to ignite interest in what you can do, fail to place yourself amongst those that are known to know about whatever it is you’re hiding. That means you can’t engage them and have them pay you for the work.

If you have a business, would you hide your products and services or describe them as fully as possible to engage potential clients? Why is it any different when you’re an IT contractor, especially if you’re working for a testing consultancy? It’s not. You need to help your client understand their options and what you can deliver. Then engage them in that and bill them.
By being open and giving away what you know, you have the potential to get more back.

What, not How.
There is a caveat here though, a little nuance in the approach that any wise consultant will understand.

When schooling your client in whatever it is you believe they may be interested in and that you could deliver – you need to tell them WHAT it is, WHAT the benefits are, etc. You don’t want to tell them HOW. That Know How combined with prior experience is why you’re needed. This is the part of the game that’s understood. Every company shares WHAT it has on offer, the exact How To is why we need them afterwards.

Don’t worry about this approach. When you present something your client is interested in, explaining your prior successes and how you can help guarantee their success – of course the first person they will look to hire to do this thing, is you. Any smart business owner will at any rate. If they’re not smart, then you’re on a route to nothing presenting ideas and opportunities anyway!

Keep an attitude of abundance, not scarcity. And share what you know, tell of the benefits and your experience, give your client options to explore; then wait for it to come back around to you.


Enjoyed this post?
Say thanks by sharing, clicking an advert or checking out the links above!
Costs you nothing, means a lot.

Image src cleverchameleonconcepts.com
Via a Free to Share and Use image search filter on bing.com

Monday, 2 March 2015

Next Gen DevOps and Software Testing

In his book, Next Gen DevOps, Grant outlines the historic path along which DevOps emerged and then describes how the way it is currently performed is fundamentally flawed. He describes a number of commonly experienced frustrations and inhibitors, both internal and external to the DevOps team, which people in other IT areas will unfortunately recognise. He shows how these impact the ability of the DevOps movement and its practitioners to drive forward their vision of what DevOps would ideally evolve into. Common issues other practice areas will recognise include; a lack of understanding of DevOps resourcing profiles by HR, a verbal agreement to the concepts of DevOps by senior management, but a fear to commit to the corporate and operational change needed to realise the vision, a continuing siloed approach that prevents the establishment of cross-functional, integrative product-based teams that are central to Grant’s view of modern DevOps practice.

Much of what Grant outlines in his book will ring familiar to software test practitioners. I and others have long espoused the value and indeed the criticality, of positioning software test practitioners as an embedded part of the ‘application development team’, ensuring cross-team process synergy1. A term I use in preference to names such as Dev team, the Test team, Ops team, Support team, etc. which serve only to reinforce the ‘silo’ us-and-them perspective. The concept of having these as teams who operate in a non-integrated way is less and less meaningful in context of today’s perspectives on efficient development approaches. Clearly defined practice domains remain important, the sheer scale of today’s IT profession requires a level of specialisation, but this is not the same as being siloed.

Taking this further, as we mature the adaptive, pragmatic, delivery focused approaches that are justifiably popular at this time, and possibly emerge into a post-Agile paradigm, approaches that were established in an era where predictive development models were overlaid onto functionaly siloed teams are, I would suggest, as good as irrelevant.

Services or Products?
Except for the most trivial of application development related work, it simply isn’t possible to deliver anything meaningful, from a technology, business or market perspective, without cross-domain collaboration. This is true because of two key factors; a) large scale application complexity is now so high, that no single person or team can perform all practices effectively and, b) the integrated nature of technology and cross-over of practice areas means that practitioners in one field will already be working in a cross-domain manner.

However, we also need to shift perspectives and come back to another key point in Grant’s Next Gen DevOps book, that rings true for us as software test practitioners and ask; are we delivering a discrete set of software testing services in support of some application development work or are we providing a suite of testing practices, alongside other practice areas, in broader support for the delivery of a software product requested by the business?

If we think more broadly than our technology domains, considering also other domains across the wider business and reflecting on why we’re employed by the business, it will be evident that we’re not really engaged in testing some development output, but instead we’re testing an aspect of a product the business has requested. With even a trivial reflection, it’s apparent that all of these practice areas for given domains need to be drawn together to support not just application development, but to support product development from conception to retirement, in context not of the Software Development Life Cycle, but instead of the Product Development Life Cycle2.

In conclusion on these limited points; while Next Gen DevOps is proposed as a model for DevOps, it discusses many concepts that run parallel to our area of concern, that of the role of software testing practice in the broader business context when delivering software products requested by the business.


Learn More...

You can get a copy of Grants book on Amazon Store
If you're on twitter, follow Grant at: https://twitter.com/grjsmith with hashtag #DevOps
While you're about it, pay a visit to his website over at: http://nextgendevops.com/ 


Enjoyed this post?
Say thanks by sharing, clicking an advert or checking out the links above!
Costs you nothing, means a lot.


[1] Crowther, Mark, (2005) “Cross Team Process Synergy” [online] Available at: http://cyreath.co.uk/papers/Cyreath_Cross_Team_Process_Synergy.pdf [Accessed 02-mar-15]

[2] Crowther, Mark, (2009) “Life Cycles – Course 1, Session 1”, pp. 3-4

Tuesday, 18 November 2014

Auto-refresh a web page

Today I discovered the main website I keep at www.cyreath.co.uk wasn't live. Shockfest! I say the main site, but it's the main 'static' site, the main site is perhaps the blog over at http://cyreath.blogspot.co.uk now. Either way, the website was (is) down and I'm waiting for support to email it's back up.

The email will be appreciated but just trying to load the site is the best way to know it's there. What I don't want to do is keep hitting F5 though. Thankfully HTML has the reload() method available. Wrapped in a little JavaScript we can use this to poll the site and avoid having to refresh it ourselves.

The script is pretty straight forward and useful for other things. You can plug in that auction site, page with a hit counter, flight status or test results dashboard page showing on a large screen in the office.

Paste the below into your favourite text editor and save it as a .html page.

           <script type="text/JavaScript">  
                function pageCheckRefresh(timeoutPeriod) {  
                //  -->  
      <!--- change this to the refresh time you want -->  
      <body onload="JavaScript:pageCheckRefresh(10000);">  
           <!--- This first url is just a control, one we know WILL be there, so we know this checker is working -->  
           <p><iframe height="200" width="750" src="http://www.bbc.co.uk"></iframe></p>  
           <!--- You can have more pages in iframes, just copy the below ( <p> to </p> ) -->  
           <p><iframe height="300" width="750" src="http://www.cyreath.co.uk"></iframe></p>  

You'll need to edit two things and add one:

Edit the refresh
It's currently set to 10000, which is 10 seconds.

Edit the target URL
Change http://www.cyreath.co.uk to the URL you're interested in.

Add additional URLs
Copy and paste the ...
section and add another URL to check multiple pages.

That's it, straight forward but handy.



Liked this post?

Say thanks by clicking an Ad


Wednesday, 1 October 2014

Ruby Selenium-Webdriver - Quick Start

Guess how old Selenium is? If you didn't know, it's now (over) 10 years old... no really! How about Selenium 2? Well that was released in July of 2011, so it's not 'new' by any means. 

If you've not had a look at it yet, now's the time! Selenium-Webdriver will allow you to execute web tests using a range of browser easier than before. You can also use your favourite programming or scripting language and a range of other tools to enhance your testing.

As always, I'll be using Ruby on Windows for this demo and assume you have Firefox browser available - let's get going!


1. Install Ruby
To do that either read the blog post here or watch the video on YouTube:

Set-up and install Ruby


2. Check your Gems
We're going to need the selenium-webdriver gem. To install that, open a CMD window (start > run > 'cmd') and type gem install selenium-webdriver. You can check installed Gems by typing gem list which shows what's available and their version.

3. Start Interactive Ruby (IRB)
For this demo we'll just run commands straight from IRB. Using a CMD window type irb to start IRB.

In IRB type require 'selenium-webdriver' to start a webdriver instance so we can pass it commands to execute.

4.  Open the browser!
Yes, we're ready to start using Webdriver. Now type the following to invoke an instance of Firefox with the reference of browser.

browser = Selenium::WebDriver.for(:firefox)

If all is OK then Firefox will open. If you get an 'access' warning, just click OK.

5.  Run some tests
Now work through the following commands to run a basic test using Google.

Type: browser.navigate.to("http://www.google.com")
Google will now load in the blank browser instance.

Type: browser.find_element(:name, "q").send_keys("Hello)
This will type 'Hello' in the query text field, but not return it.

Type: browser.find_element(:name, "btnG".click)
We'll now see search results returned.


Watch the video!

References for DYOR:


Liked this post?