Yet another bloody blog

A thought on learning

2012-01-13T02:48:00.001-08:00

Over at the Software Testing Club I blogged about getting stuck learning coding and how it was a frustrating experience. Last night I had a good conversation with my other half about how she hits the same issues studying her accountancy course.

She described the same issues, the study seems to be making sense and then boom... a total lack of understanding about what's just been read. Flicking back to review the previous material then reading back over what's unclear doesn't help.

It's the same with me and code. I'm merrily hacking away, currently over at Code Year by Code Academy and through Brian Marick's Everyday Scripting with Ruby, then my head implodes or I run out of brain sugar or something. What I've just read no longer makes sense, in fact it didn't make any sense but clearly is meant to.

So what's the problem? We talked about if for over an hour and came to a couple of conclusions.
Issues such as limited intellect, boredom with the subject, lack of sleep, lack of a good study environment aside, we agreed the core of the problem may be two fold.

Firstly you have a unique collection of experience and current knowledge. That includes experience and knowledge about the subject being studied but also about other things that might be relevant. Studying code for example might be helped by knowing languages, chords or math (so I hear...). Studying accountancy might be helped by knowledge of business, maths, analytical techniques.

Secondly, you have the material in front of you that only has the material in it that it has in it and can only be explained in the way it's being explained.

So, you read the material and you get stuck and now you're really stuck. You now have no further context from your existing experience or knowledge to apply, that might help you make sense of the material so how can you possibly progress?

This I think is common, in that many courses assume a level of knowledge that you don't have. Therefore the material simply cannot make sense to you and you don't have any context (as above) in which to interpret it so it could make sense. You're stuck and you can go over it again as many times as you like but it wont help.

To get unstuck you need to do a few things

Practice - thereby gaining more experience which will provide greater understanding and ingrain what you're studying into your brain more fully.

Find another source - look for the subject you're stuck on somewhere else and see if the way the material is presented gives you additional understanding.

Best still...

Get a mentor - a mentor combines both of the above and you skip a lot of the steps needed to get moving again (maybe a good or bad thing but sure is quicker!)

Invitation - write papers, articles or blog posts with me

2012-01-03T07:58:00.000-08:00

About once every 2 or 3 months I get what seems to be a serious approach by someone to do some collaboration with. That collaboration is usually simple enough, perhaps writing a paper or a series of blog posts together. It may be simple enough but it’s a great thing to do. I love the idea of collaborating on the research, writing, review and publication of papers with other test professionals.

But… no one and I mean NO ONE every properly follows up with their offer. There’s some initial discussion, a bouncing back and forth of ideas and then tumble-weed. I can only harass so many times. OK, I’ll cut Kashif and Zaman some slack here as discussions are technically ‘on-going’.

I do understand, there are a few reasons I can think of why papers and articles don’t get written.

Fear of scrutiny (looking stoopid)
I suspect the biggest reason people don’t write articles, papers or blog posts is the fear of peer review, critical peer review and response more importantly. We all suffer this worry, less as we get older and more gnarly sure, but it’s always still there. Let me tell you, first off don’t worry about those who would try and put your work down, secondly look for the criticism.

Do it for two reasons, one is it helps you grow as people question your thinking and understanding and you need to respond, the other is any morons trying to look smart by making smart comments soon show themselves to be the morons they are. It’s a win win for you. So get writing.

Your opinion doesn’t matter, it’s already been said.
Sure, a quick search return 100s of articles on metrics, agile, test management, tools, writing bugs, creating test data, so why bother writing yet another? Heck one you write will be less insightful and badly researched right? Well, maybe but there’s a special something that only you can bring, your unique experiences. Only you can share the from-the-wild stories you’ve experienced and the best way to write is often to tell stories and share what you learned.

What’s more there’s always a need for a new perspective on an already discussed topic. If not then websites, blogs, book publishing and YouTube would grind to a near halt. It’s never been said quite in the way you will. So get writing.

You have a topic but don’t have the answers.
I once wrote a paper called ‘Assessing the Developer / Tester Ratio’, addressing that old question. It was 10 pages of working through models based on measuring development and testing factors, project management theory of constraints, then combining them into a simple and complex model. All with the intent of assessing the required number of testers to developers. I even wrote a html/JavaScript page to do the calculations. It was awesome to the power of geek. It was also completely impractical and the end result was a recommendation the reader would better spend their time just agreeing a ratio (e.g. 4 to 1) and seeing how it worked out for them.

Was it pointless writing it and spending the time doing the research? Absolutely not, contrary to common belief you DO NOT have to have all the answers. Even a PhD thesis leaves questions unanswered. Why not research and write as far as you can and then leave it to the test community to pick up? Perfectly acceptable approach, not one expects you to be a know-it-all, so get writing.

Unt Zo… to zee challenge!
I’d like to invite anyone in the test community to collaborate on writing a paper, article or blog post.

• We’ll split the year into two month slots starting in January. So that would be Jan-Feb, Mar-Apr, May-Jun, etc. Pick a slot that works for you and we’ll get onto it.
• You can pick the topic or we’ll think of one together.
• That catch is – it needs to be a 50/50 effort, more on your side is better!

How hard can it be? What have you got to lose? Etc, etc,… :)

Mark

You don’t want to do ‘Penetration’ Testing, probably

2011-11-04T07:25:00.000-07:00

Here’s an interesting thing, when people talk about Security testing they usually talk in terms of Penetration testing. You can imagine the scenario, “We need to make sure our site is secure, let’s hire a security consultant to do some penetration testing”. However, they rarely mean Penetration Testing. It’s the same as people talking about software quality assurance when they mean software testing. It’s similar to Performance testing when they mean Load or Stress testing.

It’s not that we don’t understand why people use these terms in a slightly inaccurate way, hey they’re not all 24/7 live and breathe testers like us, we can give them a break regarding the terms they use.

The point is, Penetration testing is more correctly thought of as; “A sub-type of Security testing focused on exploiting vulnerabilities present in the underlying design, application code and functionality and the network the application is deployed on. This includes exploiting interconnected software systems, networks, access controls and physical security.“ to quote the Test Hats ‘Software Testing Dictionary’.

Another good definition comes from the book, ‘Backtrack 4: Assuring Security by Penetration Testing’, “The basic difference between a vulnerability assessment and penetration testing is that vulnerability assessments identify flaws… without measuring impact, while penetration testing… exploits these vulnerabilities”.

Here’s the punch line that gets missed, the active exploitation of vulnerabilities should be expected to leave systems in a compromised state and that means they’ll need to be rebuilt. You did make a back-up of the data, configuration, etc. right?

That previous bit is the bit that doesn’t get mentioned very often it appears. No, not the need for back-ups… the bit about Penetration testing of live systems leaving them in a compromised state. Truth is, Penetration testing proper is a risky business and a service that most clients don’t really want and don’t need.

Most often what’s needed and what’s being asked for is an audit and assessment of how secure the network and application is. Of more value to them is seeing how robust their ‘security posture’ is in terms of technology, configuration, patch cycles and so on. They can then take a Vulnerability Assessment or Audit report and remediate any issues on there. This is far more valuable than Penetration testing on its own.

However/But… because there’s always a but… Penetration testing has its own unique value in that if you want to truly harden your systems, your need to do Penetration testing. The fact is that Compliance Audits, Patch Audits and Vulnerability Assessments are, in the main, checks. Audits are checks of compliance against a given standard, such as PCI-DSS or your own standards, they’re fairly static. Vulnerability Assessments are more dynamic, given the threat landscape changes day to day it isn’t possible to statically define some threats in an ‘audit file’ and merrily check them.

Penetration testing is not only more dynamic the testing approach required is far different too. This is testing, not checking or auditing. But… that’s a topic for another time!

Mark

Best Practices - Smoke and Mirrors or Cognitive Fluency?

2011-11-03T04:18:00.000-07:00

Simon Morely made reference to a blog post I wrote over at the Software Testing Club recently. It relates to Best Practices and a conversation we had in house here at Test Hats. He makes some great points and the post is well worth a read.

The Tester's Headache: Best Practices - Smoke and Mirrors or Cognitive Fluency?

The original blog post I made is linked to from his.

Mark.

Why I added Lisa Crispin and Gojko Adzic to Wikipedia

2011-10-13T05:05:00.000-07:00

Wikipedia is an encyclopaedic store of knowledge about the world, the people and events that have shaped it in the past and are shaping it now. The wonderful thing about Wikipedia is it contains knowledge about groups in the world, one group is the software testing and development profession/industry. This is relevant because software testing and development is at the heart of how the modern world is being shaped. The world and even those things we send out of this world all run on software. I want you to realise the significance of this and the importance of software testing before I gone on…

We are in the computer age and it’s as historic a moment for the world as the steam age and industrial revolution was. Yet, are we recognising this and ensuring we capture our knowledge of those that are shaping it?

Software Testing People

If you look at Wikipedia there is a category called “Software Testing People” (http://en.wikipedia.org/wiki/Category:Software_testing_people) and it’s a pathetic list of 15. (16 as I’m there, my user page is labeled to appear). We rightly see James Bach and Cem Kaner. I recognise Rex Black, Boris Beizer and Brian Marick. The rest? Never heard of them and I question whether they are really ‘testers’, but that might be showing my lack of ‘education’.

That aside, are you telling me that’s all the people who have or are causing paradigm shifts in the way we think about software testing? In how we approach it and re-shape the profession? Can you tell I’m offended and getting precious about my beloved profession? .

That’s why I’ve added Lisa Crispin and Gojko Adzic, I’ll be adding Michael Bolton too when I get time, please go ahead and beat me too it! These people have literally affected the entire profession. They have on their own and in combination with others (and the test community) caused a paradigm shift in the way we think about and do software testing (and from it, development). They have been around for years, are known and respected globally. Will be quoted, emulated and recognised (along with Bach and Kaner) as thought leaders and pioneers for as long as this profession exists.

You bet they should have Wikipedia pages!

http://en.wikipedia.org/wiki/Gojko_adzic
http://en.wikipedia.org/wiki/Lisa_Crispin

Testers coming to London

2011-10-05T00:15:00.000-07:00

I get about 2 emails a week via the www.testhats.com site from testers looking to come to the UK, asking how best to do it. I also get emails about sitting ISTQB exams but that's another post.

I'm of the biased opinion the best place in the world for a testing career is London. If you are serious about a successful career in testing, then making the move there is a wise choice. One problem you will have though is there are many testers in London and some of them are very very good. Some of them are crap too, but you don't need to worry about those.

Therefore you need to do two things:

• Make it easy for potential recruiters to find you and learn about you.
• Differentiate yourself by becoming ‘known’ for your testing knowledge and views

In order to prepare yourself for arriving in the UK, you may have done this already but in case you've not here’s some things I’d suggest;

• Head over to http://www.softwaretestingclub.com/ and start interacting with the test community, it’s mainly UK based so will get you exposure
o Join relevant groups and interact: http://www.softwaretestingclub.com/group/uksoftwaretesters, http://www.softwaretestingclub.com/group/uklondon
o Set-up your blog there if you don’t have one, post at least once per week

• Join http://weekendtesting.com/ and the sister activity of Week Night Testing, then participate.

• Set-up your Linkedin profile - http://www.linkedin.com/ and get connecting to people
o Link-in with me: http://www.linkedin.com/profile/view?id=6436250 (Choose that you’re a member of STC)

• When in London be sure to attend the London Tester Gathering and meet the London test community
o http://www.linkedin.com/groups/London-Tester-Gathering-2656070

• Get a twitter account and follow other testers. Start to see what people are talking about and join in the conversation.

The most important thing you can do is participate in the testing community, either online or in person. The second most important thing is to start formulating a clear idea of your views on testing. For that you’ll need to study and practice, read and debate, then share your ideas.

You own your career and are to be congratulated on your determination to further yourself as a software tester. Please remember the correct order of things is Family > Work > Church and pay attention to them in that order.

Finally, I'd wish you good luck, but as I concur with Seneca who said "Luck is when preparation meets opportunity", I'll simply say "Get ready, then get on with it!"

Mark.

Security Testing Research, links galore

2011-08-30T05:07:00.000-07:00

Over at the Software Testing Club I just added a list of resources for use by members of the Security Testing Group I set up. I thought I'd add the list here for reference and encourage readers to visit the STC group.

Websites and Forums

Dark Reading: http://www.darkreading.com/
Infosecurity: http://www.infosecurity-magazine.com/
Ethical Hacking Blog Site: http://www.ehacking.net/
The Ethical Hacker Network: http://www.ethicalhacker.net/

Podcasts and Video Series
Cigital Silver Bullet Security Podcast: http://www.cigital.com/silverbullet/

Security Testing Methodologies
OWASP: https://www.owasp.org/
OSSTM: http://www.isecom.org/osstmm/
ISSAF: http://www.oissg.org/issaf/

Threat & Incident Classification
WASC-TC: http://projects.webappsec.org/w/page/13246978/Threat%20Classification
WHID: http://projects.webappsec.org/w/page/13246995/Web-Hacking-Incident-Database
Taxonomy of Coding Errors: https://www.fortify.com/vulncat/en/vulncat/index.html

Tools

Backtrack: http://www.backtrack-linux.org/
NMap: http://nmap.org/
Nessus (Home Feed): http://www.tenable.com/products

Hack to learn, dont' learn to hack.

How to get started on SQL Injection

2011-08-09T07:55:00.000-07:00

Firstly, you need a good working knowledge of SQL. That may seem obvious but you can't just rattle off a bunch of SQL strings and have no idea what they are meant to be doing, what they are testing for and expect to test well.

Head over to here and diligently complete each of the exercises:

http://www.sqlcourse.com/
http://www.sqlcourse2.com/

Secondly, get some pre-cooked SQL vectors to try out.

Go to http://ha.ckers.org/sqlinjection/ and try out the vectors MANUALLY

Do them manually to learn what they are, really read them and get familiar with SQL attack vectors. Try and construct some of your own given your knowledge of the app you're attacking.

Thirdly, Open Firefox and add 'SQL Inject Me'

https://addons.mozilla.org/en-US/firefox/addon/sql-inject-me/

Play with this add-on and see how it changes how you approach your testing. When you're done go to Firefox and click on "Tools > Add-Ons > Extensions > SQL Inject Me > Options > SQL Injection Strings" and add the bespoke vectors you created earlier.

Have fun!

Mark.

Principle Test Architect, Test Hats.

Security testing noobs and OWASP Breakers

2011-03-04T14:18:00.000-08:00

I’ve said it before but one area of testing that functional/system type testers really neglect is Security Testing.

The neglect comes in two forms, one is thinking they know stuff when in fact they don’t. Most will rattle off SQL injection as the security test technique they know and it’s so ignorant and arrogant it makes my blood boil.

The second form is assuming security testing is the domain of rocket scientists types with PhD’s a l33t skillz in math. While true in some regards it isn’t the full story, just like SQL injection isn’t either.

If you’ve been in either camp I want you to stop before you do harm. The only way to do effective security test is the same way we do effective functional/system testing, diligent and consistent study and practice.

If like me the majority of your testing is against web sites and web applications then you need to head over to the OWASP website right away. That’s the ‘Open Web Application Security Project’ and the website has a host of free information and guides you can learn from.

Check out Michael Coates blog too, he’s trying to get a more active community going and one group is the Breaker group. As a tester go sign up and start to interact with the security testing community.

Get your skills honed and add real security testing to your arsenal of testing types.

By the way, Injection is number 1 on the OWASP Top 10 Application Security Risks list. So well done on getting that one. Number 2 is Cross Site Scripting (XSS), do you know how to do that? Here’s a vector for you:

Only 8 more to go!

Mark.

London Tester Gathering - May 2010 - Sign-up now!

2011-02-28T04:57:00.001-08:00

I just looked over the growing list of talks and workshops being given at the London Tester Gathering in May and bloody-hell it's looking good. Not sure how Tony manages it but these gatherings just go from strength to strength.

http://skillsmatter.com/event/agile-testing/london-tester-gathering-2011

Given I have a lot to say about testing I fancied I'd request a slot to talk. But... I'm going selfish on this one as I want to be in the audience and learn from some of the folks doing the presenting and workshops.

There's already too much cool stuff happening across the two days to see it all!

If you do nothing else today go check the site out and sign-up. Book holiday, coerce your boss into giving you time to attend - just do what you have to do to be there!

http://skillsmatter.com/event/agile-testing/london-tester-gathering-2011

When you've read it please copy and send the Tweet below:

London Tester Gathering, May. Sign-up now! http://skillsmatter.com/event/agile-testing/london-tester-gathering-2011 @tonybruce77 #LTGMay RT!

Mark 'is it may yet' Crowther

Usability testing - a little checklist to use

2011-02-15T07:12:00.000-08:00

Looking at the UI it wasn’t clear to me where the buttons were. That might sound odd but the graphics of the website were very ‘arty’ and it wasn’t immediately obvious where I was to click.

I remember another website I visited where I added items to the basket but wasn’t sure if they had been added. I then wondered if in fact the ‘add to basket’ button was inactive, maybe I needed to log-in first.

I’m sure we’ve all been on websites where the layout changes and our tour of it is suddenly halted while we work out how we use the site.

These are all examples of usability issues and though we as testers often know almost instinctively that this makes sites and applications ‘buggy’ but we might hesitate to raise them as bugs. The closest we come is probably functional testing with an uncertainty around whether these types of issues go beyond what we should be testing.

This is in part I think because as testers we don’t have a clear idea of how to classify Usability issues and in part because we might be stuck in a ‘requirements validation’ mind-set.

Getting out of just validating requirements is already being talked about a lot in the testing community so we can take that one as read. Having a clearer view of how to classify Usability issues we can talk about.

When I write Test Plans I have the usual system, functional test types but also Accessibility & Usability. I put the two together because many Accessibility considerations are affected by the Usability of the site.

When doing Accessibility testing we’re thinking about people with various levels of visual, physical and cognitive impairment that would benefit from a little ‘reasonable adjustment’ of the way we create software.

Usability is connected to this but extends beyond it, affecting every user of the website or application. Therefore this under-addressed form of testing needs to be factored into our test planning.

My guidance has come from the simple list below, courtesy of the Open University. When you experience a Usability issue in the software you’re testing, but you think it’s stretching the definition of a requirement – just label it as one of the below. Get the bug tracking software set up with a Usability classification to. Oh, and go chat with your company’s User Experience person and talk about this. One more team on our side :)

HCI Design Principles
• Visibility in the context of UI design means making it clear what a UI element is used for.
• Feedback is related to the concept of visibility. In the context of UI design it means making it clear what action has been achieved through the use of the UI element.
• Affordance in the context of UI design means making it clear how a UI element should be used. Everyday examples include a cupboard handle to be pulled and a door plate to be pushed. When the affordance is wrong, a written sign is often needed.
• Simplicity in the context of UI design means keeping things as simple as possible
• Structure: A UI will be more usable if it is structured in a way that is meaningful and useful to the user.
• Consistency in appearance, positioning and behaviour within the UI makes a system easy to learn and remember.
• Tolerance refers to the ability of a UI to prevent errors if possible, or to make them easy to recover from, if not.

Source: The Open University, M150 Unit 12, ISBN0749257695

More information:
This book makes interesting reading: http://www.amazon.co.uk/Design-Everyday-Things-Donald-Norman/dp/0262640376
Website Design Considerations for Older People: http://www.usabilitynews.com/news/article2911.asp

Tools:
Web Accessibility Toolbar: http://www.visionaustralia.org.au/info.aspx?page=614
Colour Contrast Analyser: http://www.visionaustralia.org.au/info.aspx?page=628
Colour Blind Web Page Filter: http://colorfilter.wickline.org/
Colour Scheme Generator: http://wellstyled.com/tools/colorscheme2/index-en.html

Test Early and Often

2011-02-02T03:22:00.000-08:00

On a recent project it was agreed that testing would mainly be executed at the UAT phase, I was to build a set of test cases that could be executed by the customer. While some testing might get done informally before UAT that was to be the main test phase. When UAT commenced there was delay while the test cases were proven and understood, before being run in anger.

Some test assumptions were wrong and test cases had to be updated. Some functionality wasn’t implemented as described in the requirements documents meaning more clarification and correction. Server issues further delayed the test progression and overall finishing UAT was delayed by around 2 weeks. Sign-off was achieved but it was done a little grudgingly based on contractual terms being met.

In an earlier project testing began before we even had the code. Test cases were used as a way to validate requirements and gaps were found. Not only in the thinking around the requirements but the third party applications that were part of the overall integration. Test cases revealed conditions that those applications wouldn’t be able to meet and a mitigation approach was agreed.

When new development was released tests were run straight away in part because automation was in place to assist. When UAT came around and the customer expected a final test run from us – we didn’t bother. There was no need for a final test run. We handed the code and test over and less than a week later got sign-off for the delivery.

These two very different approaches highlight the importance of testing often and testing early.

It’s not just about ironing out the issues early on. We want to do that of course. We want to find the issues in the requirements, in our test cases and assumptions, we want to have clear information in developer hands so they can code once and quickly. (Developers are artisans and artisans hate reworking what they’ve produced, they’re proud of their work like we are.)

Testing early and often saves time (and money), it cuts frustration and keeps people motivated as it gives feedback and insight, and confirmation of a job being done well.

Customer confidence and belief in the quality of work we produce, our ability as a competent and professional team all come across when we test early and often. In consultancy this is the holy grail of keeping customers and winning new ones. So many consultancies just do the job and follow the contract, it’s immoral, think of those that have been sued in recent years.

Build customer confidence by making quality and delivery a foregone conclusion by testing early and knowing it will be.

You’re better than you think you are

2011-01-26T03:20:00.000-08:00

Many software testers I’ve worked with, especially more junior staff, and not just to me as that’s getting easier every year, seriously underestimate themselves. One thing that’s helped me in my careers is an over developed sense that the work I produce is completely awesome. Accepting that to be true (see what I mean :p) it doesn’t mean that it’s ‘better’ than anyone else’s work.

James, Michael, Gojko, Pradeep, all do cracking work that’s impressive but some of what I’ve done matches it. It’s going to, they’re them and I’m me and we all have our own unique combination of skills, abilities and experience. That means I’ll come up with approaches and solutions that they could never think of, their own work speaks of a similar thing. Whether they have more public awareness of their work or not also doesn’t detract from how damned good my stuff is.

It’s the same for you too. You’re better than you think you are.

Everyone who has a passion for testing, is on the forums, chatting to other testers, reading or writing blogs, asking questions, going to conferences and gatherings and just ‘engaged’ intelligently and actively in the profession is as good as anyone else. By good I mean your ideas, thoughts, solutions, etc are as useful, insightful and ‘right’ as anyone else, known in the test community or not.

I want to encourage anyone in the profession who thinks their ‘ok’ at what they do to find the one thing they feel strong about. Maybe it’s writing test cases, using Exploratory testing, adding a splash of Ruby to the tools you use or some other thing you know. It doesn’t matter what- find that one thing and realise – you’re better than you think you are.

Take a deep breath, have courage and go and tell people about your one thing. Perhaps it’s a blog or forum post, maybe a 1 hour Skills Matter talk, a YouTube video or a paper/essay on your thing. People want to know and the test community is a very accepting and supportive community. They will share with you the fun and excitement of what you’re sharing with them.

It shouldn’t be a big event to speak at a conference, write a paper, present at a tester gathering. Get used to it, start now and watch how you grow as a person and a professional.

Everyone of us starts on the first step, sometimes we don’t feel too clever or have to admit we only really know our one thing. That’s fine, better to say that than try and act like a know it all. Some of my best learning and growth experiences have come about when I said ‘I don’t know’.

Too many junior testers are staying out of the limelight and you need to get in it. Sharath is a good example of someone doing generally good stuff and discovering he was better than he thought he was. Take his lead and get yourself out there, helping yourself and the test community at the same time!

What is Given-When-Then anyway?

2011-01-10T12:17:00.000-08:00

I’ve been asked several times what a working set of Given-When-Then examples looks like in practice. The full style is not just the GWT component, though I have used that on its own. In full we might be having a Scenario followed by one or more Features with one or more Examples.

Here’s what a set might look like, based on some recent testing of Microsoft Office Communicator Group Chat. However, this could apply to any chat client or forum board.

You’ll see we have one Scenario that is the goal the business wants to achieve. To achieve it the software has to have a collection of Features (aka functionality) and to prove those Features are there we have Examples (of behaviour the system can enact – my definition of Functionality*)

Hopefully you’ll see how ‘obvious’ they are to any reader in any team due to the narrative format.
--
Scenario: Once the user has joined one of the Chat Rooms available to them they need to be able to read existing messages. These can include text, links, stories, files and images.

Feature: A user can see persisted and new messages in a Chat Room they have just joined
AS A user who has just joined a Chat Room
I WANT to see messages other participants are posting now and have posted in the last days
SO THAT I can read messages shared in previous days or being shared right now

Example: The user enters a Chat Room that has several days’ worth of messages
GIVEN the Chat Room has messages posted over the last days
WHEN I Join the Chat Room
THEN I can see the messages for the last days with a date stamp against them

--
Feature: A user can view attachments, links, images and stories contained within messages
AS A user viewing messages that contain more than simple text
I WANT to be able to open them and see the contents
SO THAT I can view the additional information and understand the message fully

Example: The user opens a message that contains a file attachment or story
GIVEN the message contains a file attachment or story
WHEN I click the message link to open the message
THEN the message opens and shows the attached file or story

Example: The user opens a message that contains a link
GIVEN the message contains a link
WHEN I click the message link to open the message
THEN the message opens and shows the link as an active hyperlink

Example: The user opens a message that contains an image
GIVEN the message contains an image attachment
WHEN I click the message link to open the message
AND hover my mouse of the image file posted in the message
THEN the image can be viewed as a thumbnail

TWIST Podcast with Matt Heusser

2010-12-13T03:30:00.000-08:00

A few weeks ago I did a 'This Week in Software Testing' (TWIST) interview with Matt Heusser. We had a great chat about the wider context of software testing, the influence of manufacturing thinking, impact of standards such as CMMi and ISO, organisations such as ISTQB etc.

http://www.softwaretestpro.com/Item/5022/TWiST-23---With-Mark-Crowther/

The site requires a free sign-up to get the podcast but with this and other podcasts it's well worth it! Have a listen to this podcast and share your thoughts here on the STP website.

Mark.

Skills Matter - Experience Report on Specification by Example

2010-12-08T09:33:00.000-08:00

Thanks to those that were able to attend last night and listen to my Experience Report on Specification by Example. (http://manning.com/adzic/)

I understand the video will be posted to the URL below in the next few days:
http://skillsmatter.com/podcast/agile-testing/specification-by-example-an-experience-report

I've asked for the slides and paper to be posted on the BJSS website and will send the URL when they're published. On side note and as I was asked; the BJSS Enterprise Agile book I handed out copies of IS available as a PDF: http://www.bjss.com/BJSS%20Approach Link at the BOTTOM of the page.

Finally, some good questions came up and I thought I'd just touch on some of them again to give slightly clearer, more complete answers.

Thanks again, it was good fun!

....................

In Concordion we have an orange 'to do' status, I was asked where that had come from

I understood it was a patch off the internet / Concordion community but asking today it appears one of the test team implemented it. I've mailed the heads of development and test to ask if we can publish this tweak somewhere (Github perhaps) so it can be available for the testing community. I'll let you know how this goes.

One Example had 'THEN: Function returns Historic Rates and Projection correctly'

The question here was how are we defining correctly. Checking today I'm told that correctness for this Example is defined in an external customer document that describes the business processes in detail. Comments mentioning it are in the code of the test where an example of what 'correct' looks like is also provided. Correct, As Expected and other phrases like this are things to watch out for!

I mentioned I'm expecting we won't run all Illustrative Examples, but if so how can we make sure they haven't become deprecated?

Currently tests against a new build take 13 minutes to complete and we have about 2 or 3 new builds a day. The next work package is twice as big as the current one. So, rough maths suggests it could take 40 minutes to run the tests in the next work package. Doing that three times a day could get painful and there could be up to 5 work packages!

As a minimum we'll always run the Key Examples. I suggest we'll look for Illustrative Examples (regression checks at that point) that have never failed or raised bugs and stop running them. Then at some agreed cadence, every third build overnight, once a week at the weekend, etc. we'll run the full suite again. It'll be a call on how much change has happened, are we approaching a drop date, etc. but I see the answer being as simple as this.

A point about how automated tests might be affected by requirements change and if we were adding Illustrative Examples wouldn't that increase the admin/maintenance burden

I re-counted the number of tests today and looked at how many had been run. I stated there were 600 tests in place, just for accuracy we have 581 automated tests for Key and Illustrative Examples of which 473 have been run to date. The total number of Examples that have required re-write due to requirements change after Collaborative Specification has taken place so far is estimated to be 8 and the number of automated scripts affected is 2. We'll see what the final figure is when the work package is complete but this is insanely low compared to traditional approaches.

Mark.

Burndown - are we tracking the right things?

2010-11-19T03:00:00.000-08:00

I saw a tweet this morning by Mohinder (@mpkhosla) that pointed to an article that made my blood boil.

http://www.infoq.com/news/2010/09/Sprint_Burndown

I must have missed the tweet, blog post or some other place where I was to learn that tracking Burndown should be done in hours. Why do I think that is influenced by traditional project management? I’ve always tracked Burndown by tasks. Tracking Burndown by hours is pointless, it tells us nothing about the actual delivery of usable functionality within a given sprint. How do we deliver usable functionality? One bit at a time, so we care about the delivered bits, those bits being delivered by completing tasks, not by working a number of hours.

Hours are just a sizing tool to see how many bits can be done in a given sprint of a certain length, say 2 weeks. That would give us roughly 80 hours per person, 64hrs if you want to plan at 80% efficiency, work it as you will. But if I log 80 or 64 hours how much work have I delivered? If you ask me in these terms you’ll want to know other details such as how many test cases I managed to prepare, automate or execute. As that’s what we really care about let’s track that. You know I’ll be here this week, ill or on holiday and tracking that is a project management type activity, an admin task, that is done away from the Burndown so don’t even think about this aspect in relation to your Burndown.

When planning testing tasks I have the team estimate the testing effort required to test a given item. We estimate in hours. When possible (when the Client ‘gets’ it) I add our test estimation to the task card next to everyone else in the development team*.

On the Burndown horizontal axis we can plot the number of days in the Sprint, in the vertical (y) axis we can plot the number of tasks we’re looking to complete on a given day. We know how many we can fit in the given Sprint days because we can divide the total hours in the Sprint by the hours we’ve estimated for the task. Math is great.

Now each day we can agree what tasks to work on and deliver them...or not. We track tasks complete/items delivered, not “I came into the office and did some work therefore that should automagically represent progress”
nonsense.

I'm sure you've seen a Burndown but just for context I'm meaning this one that I create in Excel, if I'm needed to create one that is.

* The Development Team is not just Developers, it’s all the other teams such as; Client, Business Analysts, Technical Authors, Developers, Testers, Build, Operations, Support. All are involved in the development of
working software that solves and continues to solve Client (customer) business problems. Therefore they all have a say in how long their tasks take that when completed will get a bit of working functionality done-done(done{done}).

Linux Intro - a rough and ready guide

2010-10-17T14:35:00.000-07:00

It may come as a surprise, heck perhaps even a shock - but I confess to being a complete Microsoft addict. I decided to crack my shell (turn over a new leaf, etc) and get a bit more into Linux. My work and home PCs now sport Ubuntu and to get to know it I spent the weekend learning-some-stuff.

For anyone reading that has been thinking to have a look at Linux - here's my Rough and Ready Guide to Cracking you Linux Shell! (ahh... witty tech references ;)

ENJOY!
---------------------------------

The instructions assume you're on a PC.

1) Get Linux
The first thing I did was install Ubuntu http://www.ubuntu.com/desktop a free Linux based operating system. After installation your PC will have a dual boot menu where you pick Windows or Ubuntu.

2) Learn what Linux is
With that installed I spent a few hours going through this Linux tutorial via a terminal window (DOS box) in Ubuntu.
http://info.ee.surrey.ac.uk/Teaching/Unix/

If you've used DOS commands in a terminal window before you'll get what this tutorial is going on about.

3) Do another tutorial
Similar to the one above but another perspective but repetition helps the learning. Skip the first few lessons, review lesson 4 for context then bounce straight to Lesson 5.
http://www.linux.org/lessons/beginner/l4/lesson4c.html

4) Learn Some Bash Scripting
In summary, take the above basic commands and rock onto something a bit more useful.
http://www.linuxconfig.org/Bash_scripting_Tutorial

Let me know how you get on!

Mark.

Free Testing Workshop, Málaga - 7th:9th Oct.

2010-09-26T13:46:00.000-07:00

I'll be in southern Spain between 7th and 9th of October visiting a client in the Málaga area. It would be great to have a meet up with some testers and do a few hours workshop on something of interest.

If you're in the area or nearby and can get to Málaga or you are a team/group no more than an hour away give me a shout.

You can set the topic and we'll split the time 50/50 between tuition and hands-on practice. Afterward we can co-author an experience report for the test community.

Here's some ideas of what we could go through in 2-3 hours.

[Exploratory Testing]
* What it is and what it's not
* Test Charters, Diaries and Sessions
* Test design techniques, Heuristics and Memes
* Reporting; Bugs and Progress
* Deploying Exploratory testers in development teams
* How to use it on it's own and in combination with other approaches.
-- I'll leave you either copies of template documents or software to use

[Live Documentation / Active Specification]
* Why documents are dead, literally!
* Live documentation; what is it and why you should use it
* Active Specification; what that is and how it's the link to automation
* FitNesse and Concordion; working example frameworks using Ruby and Java
-- I'll leave you the example FitNesse and Concordion frameworks

[Selenium (Ruby) Automation]
* What are the Selenium tools and why we use each
* Getting set up with IDE and RC^
* Using the IDE for assisted testing and rapid prototyping
* Converting IDE tests to Ruby for use with RC, to test across multiple browsers
- I'll leave you with a working Selenium-Ruby framework and some sample scripts

^ I don't think we'll have time to get Grid installed and running but I'll leave a workbook of instructions I have so you can follow up.

After we discuss each we'll get hands on and do some testing activities. You should end the session being 'able', not just having theory!

Contact me!
Email: mark@cyreath.co.uk
Linked-in: http://uk.linkedin.com/in/markcrowther/
Twitter: http://twitter.com/MarkCTest

Mark.

One Measure to Rule them all

2010-09-25T05:29:00.000-07:00

Measures and Metrics are back in discussion again in the Mark camp of testing. I recently met with Mohinder Khosla around St Pauls after we’d exchanged emails on the topic and shared some material with each other.

I recently wrote a Test Approach where I included five of my favourite measures:

Test Preparation
• Number of Acceptance Criteria v. Number of Test Cases per functional area (Bar Chart)
• Number of Test Cases Planned v. Written & Ready for Execution (Burndown)

Test Execution and Progress
• Number of Tests Cases Executed v. Test Cases Planned per functional area (Burndown)
• Number of Test Cases Passed, Failed and Blocked (Line Chart)

Bug Analysis
• Total Number of Bugs Raised and Closed per period by Severity (Line Chart)

What that struck me as I was writing the approach was that we really needed to know one thing:

“The total number of Acceptance Criteria moving into a pass state – this is PROGRESS”

Sure we need ways to assess the backlog, complexity, test execution progress, etc. but as an ATDD project we want to know what acceptance criteria have gone green.

Mohinder is working on the paper that includes a number of sources and views, look forward to seeing it soon.

Test Case Workshops

2010-09-17T06:21:00.000-07:00

Hidden deep within Requirements and Functional Specification documents are the test cases we’re looking for. Like any hidden treasure they need to be discovered by effective exploration, deduction about the nature of the environment they exist in and who put them there and why.

When we (testers) review theses document or other sources that provide this information we have one question at the front of our mind, ‘what test cases does that need?’ We need to be thinking how we’d know that a requirement or acceptance criteria has been delivered on via the software functionality that’s been developed.

How are we going to quickly and efficiently find those needed test cases? The easiest way is to brainstorm them in a workshop with testers.

Get one or two other testers with you and a collection of requirements or acceptance criteria you want to find the test cases for. Assign one to write on the whiteboard, another to write up the test cases and another to perhaps just freely participate and shout out ideas. Everyone is equal, all ideas are to be explored, no negative comments or limiting of creativity and ideas.

As each item it talked through them up on the whiteboard, use flow diagrams, spray diagrams, UML type layouts, mind mapping – whatever allows a free form diagram of linked ideas to be created. Each idea is a possible test cases. Let’s see what we might draw up and talk about if the requirement was for a user to log-in to a system.

EXAMPLE
The user logs-in, so they must be able to logout. If they log-in there must be some log-in information such as a user name or password, Q) what’s the structure of the user name and password? Where are they stored? What characters are / aren’t allowed? How many? Where are unsuccessful log-ins recorded? How many failed log-ins are allowed? Etc, etc.

After more questioning and reading around the requirements, Functional Spec and acceptance criteria a diagram might look something like this:

Test Cases
From here we can start to capture test cases based on what we’re thinking. In the format I prefer these days they might include:

  GIVEN a user has an active account
  WHEN they log-in with a valid Username and Password
  THEN they are authenticated and an active session is started

Some might not be so tidy as they require multiple outcomes such as:

  GIVEN a user has an active account and the failed log-in count is 0
  WHEN they attempt to log-in with an invalid Username and valid Password
  THEN they are notified of a failed attempt
  AND the log-in failure is logged
  AND the failed log-in account is incremented to 1

This is fine, it’s still one test condition it just has multiple outcomes which is more like reality in complex software.

Have a go at writing some more test cases against the above diagram. Have we missed any other test conditions?

We’ll leave the idea of Concrete Examples, Analysis Techniques, etc. for another post.

For now reflect on the complexity of the above Log-In example, even if it seemed simple at first. Think about how much easier it is to brainstorm the test conditions with your testing colleagues and get the test cases written up in the same session.

Take a break, share the test cases with others (Dev, PM, etc.) and get their take on them. We also had a lot of questions so share those out and get clarification on them. This approach is much more efficient than one person being sat at their desk alone. It means knowledge sharing in equal terms and ready progress.

Bugs, why are we raising them?

2010-08-17T10:08:00.000-07:00

It’s troubled me for while – that we’re not clear on why we’re raising bugs. Sure, to get them fixed we say but there’s a whole bunch of ulterior motives.

One is to show how good we are as testers as we need to demonstrate how clever and effective our testing is right? I’ve pondered before whether a test case that doesn’t find bugs during the development/testing phase was a useful test case. That’s another discussion for another post.

A second reason to raise bugs is to beat developers over the head with them. Isn’t that how we often say it too? In partially comprehensible terms when we really mean it’s to show the developers they’re not as good as they thought and make sure they see we’re as good at what we do as they are at what they do. Pathetic point scoring nonsense.

When we raise a bug we’re identifying a task for a developer to work on. They work on it, the quality issue is resolved, proven to be so by our re-testing, rinse and repeat and we all help deliver working software. That’s why we raise bugs, to record tasks that need to be done and help deliver working software.

Now we can do other clever things such as creating bug taxonomies and carrying out root cause analysis to address systemic quality issues, improve productivity and reduce costs. However, in my experience that’s never done, memories are too short to care. Find the bugs, fix them, deliver working software in collaboration with your project colleagues not in confrontation with them.

Mark

Specification by Example and Agile Acceptance Testing

2010-07-05T14:10:00.000-07:00

I recently attended a great workshop with Gojko Adzic the other week covering 'live documentation', the workshop was called "Specification by Example and Agile Acceptance Testing", check it out on his site (http://gojko.net/).

At the testing consultancy where I work we already routinely use a couple of tools for live documentation as part of the 'Behaviour Driven Testing' (BDT) approach I developed around BDD.

Live documentation tools of this kind include Cucumber, Concordion, FitNesse, SpecFlow, etc. (links below) and provide a way to help ensure the documentation is current and up to date. They get away from the 30 page Word document that's out of date and a nightmare to keep under
version control.

I'm a big fan but wonder how used these tools are over traditional approaches, so my questions:

* Are you using any of these?
* If not why not, if so what drove you to use them?
* If you are using them have you taken the next step and used them to drive automation?

Mark.

http://cukes.info/
http://www.concordion.org/
http://fitnesse.org/
http://www.specflow.org/

Update for 2010 or Oh, there you are.

2010-02-14T02:04:00.000-08:00

It's been a few months since my last blog because I've been seriously distracted by 'other things'.

One of those things was the Selenium-Ruby book and associated framework and code examples. The process of writing it and deciding which content to add has been fraught with trip wires. It's been the subject of a number of conversations with my employer and those in the industry who've helped me understand Selenium and Ruby. The main and valid concern of these various groups being whether I was going mercenary on them and heading off to claim the glory with no particular forethought for their claim or contribution over the material. That was a slightly unexpected response that derailed my effort and motivation somewhat to say the least. The writing progresses but has changed direction more than once.

Including Ruby has been fun and interesting and the great thing is it has pushed the boundaries of the 'S**t I don't know' and 'don't know I don't know'. Remember that wiring Ruby in there is to get away from the IDE as fast as possible. Not that the IDE is crap, just that it's limiting. That's another interesting point. The excitement around my employer's (NMQA) growing number of engagements using Selenium and Ruby have brought about another change. They're about to announce super clever things around Selenium and Vienna. Vienna Studio and VSL. Watch the NMQA website. I'm testing it now and it'll be Beta within weeks, I expect the Firefox Selenium IDE to become a rarely visited friend. When you see the new NMQA website, you'll see what Vienna Studio and VSL are all about.

Meanwhile back on home turf I've been studiously avoiding all forms of blogging and forum posting and just tweeting occasionally. Why have I not been forum posting? Two reasons, a) The same old basic testing questions coming up again and again from different people, b) pseudo philosophical threads of conversation that roll on for weeks at end and could be finished in two paragraphs. I've little to no interest in b) because I don't have the time or see any real practical value in a three week conversation that's a definition about two words. It's been going on for months in more than one forum by more than one person and must have some of the new comers to the profession wondering what we're on. While we're still not able to resolve issues of a) you'll have to excuse me while I ignore the b) threads. This isn't a dig at individuals it's a dig at a) not being resolved and the focus of our energies to get it resolved.

Why is a) so annoying to me? You wouldn't find a Surgeon, Aerospace Engineer or Solicitor asking basic questions and that, as you probably know, is exactly the level where I think our profession should be. There have been attempts before to resolve a) and I wish I had the time to start some big-idea-to-fix-it but I doubt I do and I doubt we can en masse. In fact I know I don't / we can't but I do have a cunning plan. A cunning plan that will perhaps allow me to stop feeling massively demotivated at the thought of trawling websites and seeing the same old tired questions. The cunning plan is a personal mentoring programme with a record of study behind it evidenced by published papers. Details to follow

An Introduction to Web Test Automation with Selenium and Ruby

2009-09-01T09:07:00.000-07:00

An Introduction to Web Test Automation with Selenium and Ruby

As readers of this blog and other random ramblings of mine across the internet will know - for more than 8 months or so I’ve been working a lot with Selenium and Ruby. It’s been a joy and a pain at the same time. Beguiled by how easy the Selenium tools were to use I got started only to discover that they weren’t as accessible as they at first appeared. A definite case of ‘easy when you know how’.

It seems there are a good number of folks out there using the tools to varying degrees and different ways which unfortunately adds to the confusion. It confuses because most folks taking the first steps with the Selenium toolset soon hit their first issues which seems commonly to be how to set the toolset up, which language to employ and even how to get over the many gotcha’s that Selenium has waiting for us.

Trawling over the forums, not just the smaller community forums either but the ‘official’ forums too, was not that helpful overall. Perhaps that’s because I was intent on asking very specific questions and/or my focus was on using Ruby. Maybe if I just wanted to use Java or buy someone’s course I’d have had better luck. In any case the lack of answers to my and other’s questions where the questions were something less than trivial was frankly disappointing. That is disappointing in a way that I feel folks are purposefully withholding the knowledge they have. There is a possibility that no one really knows too, but over five of the most trafficked forums? I doubt that. As if to confirm my deluded suspicions I even managed to find someone I would have considered a ‘Selenium Expert’ posting that they’d stopped posting - because the forums were full of ‘problem posters’. Well, I hate to spring a shock fest on you but that’s kind of the idea of public forums most of the time. FFS, can you say ‘testing community’?

So where to turn to if I’m a Selenium newbie wanting a clear idea of how to go about setting my Selenium web test framework up? Hoping for insight into other people’s success spelt out to such a degree that I can follow it step by step or translate it into my environment? I could try the official docs at Selenium HQ but it’s hardly an idiots guide, as good as they are getting as they become more complete. Where’s the guided, instructional, friendly, tutorial and walkthrough with helpful pictures and handy hints? They’re rhetorical questions and the reasons I wrote “An Introduction to Web Test Automation with Selenium and Ruby”.

Is the book an answer to all problems and perspectives you might want answers for? Not a chance. This book is an introduction to not a definitive guide and that’s the point. It’s written so that within a week the reader has a fully working Selenium-Ruby automation framework up and running which includes key process techniques for managing how it’s used. The book covers the Selenium tools of the IDE, Remote Control and Grid, provides numerous examples of the Selenium-Client keywords and commands in use, introduces the Ruby language and techniques for designing automated test scripts, it walks through the creation of 10 key test script templates and discusses ideas on managing test suites and test environments.

An Introduction to Web Test Automation with Selenium and Ruby is the book I was looking for 8 months ago and that I’ve written to capture the learning I’ve experienced bringing the framework to life on it’s now multiple live deployments. It’s my attempt to help other’s avoid wasting time that they don’t have and which needs to be focused on delivering great testing. There’s a lot to learn and it’s insane to think every tester wanting to do so is scrabbling around on forums and sites trying to piece the bits together. This book is a vehicle to kick start a more open and robust form of discussion and sharing by others in the community. That discussion will be of critical value and importance as there are a myriad of ways to implement the Selenium tools and the testing community needs to hear what they are.

Book Proposal
Having got just 2/3 the way through writing the book I decided it was time to start sending out the draft manuscript and see what publishers make of it. The first to get site of the manuscript is www.pragprog.com and it should hopefully be reviewed this week. Let’s see what the feedback is. Once I know where things are headed I’ll check in and see what I’m ‘allowed’ to share in advance of any publication.

So please do me and the Selenium would-be test community a favour and go put the word out, An Introduction to Web Test Automation with Selenium and Ruby, is out looking for a publisher.

Mark.

Behaviour Driven Development (BDD) and the Testing Profession

2009-08-09T07:04:00.000-07:00

I posted earlier in the year about BDD. If you’ve not encountered BDD have a wander over to http://behaviour-driven.org/ and take a quick read or hit YouTube and watch Dave Astels (http://techblog.daveastels.com/) Google TechTalk http://www.youtube.com/watch?v=oOFfHzrIDPk. BDD isn’t some kind of Flavour of the Week new thing, it’s been around a good few years so hitting Google or Bing will turn up even more material.

I've read about approaches such as Agile Acceptance Testing (http://snipurl.com/pio2q) too and they offer good ideas about focusing on what’s important to the customer. What really struck home for me about BDD is the focus on Behaviour. As Michael Bolton said “... our clients don't value the code as such; they value the things the code does for them."

Behaviour Driven Testing (BDT)
There is a slight ‘gap’ in all this BDD goodness though and that’s the integration of the testing world’s perspective and practices. Especially those practices that could be applied, as is or modified, to fit alongside a team practicing BDD. AS much as I have the warm fuzzies for BDD it comes across as a far too developer centric perspective yet again. Maxim No. 27: ‘Software Development is more than just the writing of code’. What’s more, I’m a tester, as such it'd be odd to talk about me doing 'development' and having to say 'obviously I mean testing...' every time I mention the word development.

Hence why I don’t talk about BDD as such, I talk about Behaviour Driven Testing 'BDT', a methodology I developed to articulate how a test team could work in a BDD environment and consider what complimentary techniques/approaches might look like. If we’re enlightened enough to realise that ‘development’ is more than writing code and is all the tasks needed (BA, PM, Dev, Test, Support, ...) to get code out of the door then perhaps we could think just BDD but the world is a more fragmented and messy place so unfortunately we talk as if development just means writing code.

Replacing, renaming BDD?
I recently exchanged messages with David Chelimsky (http://www.davidchelimsky.net/) (and Bret Pettichord (http://www.pettichord.com/)) about BDT and a question that came why call BDD by the name BDT? A valid question where the short answer is “I’m not”. Just to be clear BDT is spawned from BDD and uses many BDD practices but the purpose, the objective is to deliver testing of the application code, not the development of the code. BDT sits next to BDD, I’m not renaming or replacing it, BDD is the wellspring that provides the framework and thinking for what BDT is.

When I read about BDD and then read the RSpec book (http://snipurl.com/pipiz) in a stroke I found the answer to many tester pains. BDD:BDT focuses testers back on Behaviour and away from the erroneous belief they are testing code (what do you think testers think they’re doing when they use Boundary Value Analysis?), it eliminates friction between developers and testers because, firstly we’re not testing code (directly) and secondly we both use Cucumber Features and now testers can write more tests (focused on Behaviour) that align with the work Development are doing, Cucumber is a catalyst to get customers, BA's and Test working together in ways they’ve been dreaming of and the first tool that allows Testers to help Developers increase velocity.

When testers then start writing edge and fail cases, end to end scenarios, etc. they like Developers can define those in Cucumber and implement them in the RSpec format. The RSpec approach ‘makes sense’ to testers as it reads so easily. If we’re not just doing manual testing combined with Selenium/Ruby it gives us a really accessible route into automation that focuses on Behaviour. With Selenium/Ruby we also get to use RSpec to structure our automation tests and kick-out neat reports too. Hang on, we’ve just found a way for Test to get really involved in (and in some aspects lead) the project, with all players, aligned with and supporting Development instead of clashing with them and we’ve got a kick-ass way to write automated tests that align with Development tests yet keep us focused on testing Behaviour not code. Is the paradigm shift this represents hitting home? ;)

With RSpec we have a way for testers to write scripted tests that mean something to them and that speaks to what they should focus on, Behaviour, not code. I suggest RSpec is as significant to the testing profession as Exploratory Testing (http://snipurl.com/pipjv). RSpec is the Aha! that smacks you in the face as hard as Exploratory Testing did and provides the paradigm shift in your tester thinking that both excites you about how you’ll work in the future and makes you lament for the pratting about you’ve been doing.

I could go on and on, point is all of the above is not (BD) development of application code. It’s an absolute focus on testing but testing with an equally militant focus on Behaviour. This is what BDT is about. It’s the testing methodology spawned from BDD that focuses testers back onto Behaviour and aligns them with the rest of the development effort. A BDD|BDT project helps testers fulfil roles the business hopes they will but until now didn’t have a way to do so. Are there other ways to approach the issue of development and test team integration, engagement of the customer by all teams in a way that appears congruent, enabling all stakeholders to contribute to each other’s success, etc? Maybe but I’ve yet to work with them.

A final thought, every time I present my BDT methodology to testers it’s great to see the impact of what it enables them to do sinks in. When customers hear about BDD:BDT, Cucumber and RSpec they ‘get it’.

Please email me for a copy of this presentation. I had to take it down as it was branded for another consultancy ;(

Mark.

What test technology and tools are you working with?

2009-08-09T04:56:00.000-07:00

I've been working a lot lately on elaborating a Selenium and Ruby web test automation framework. It's going well of itself and I've used the framework with a number of clients successfully in the last 3 months or so. As I always say - lot's more still to learn!

It came as little surprise that the use of Selenium and Ruby was not what most expected as say QTP or Java are more commonplace. This prompted me to think about:

What technologies and tools are you using now in the testing field?

From my side I'm using as tech or tools sat on my desktop:

* Selenium: IDE, RC and Grid
-- http://seleniumhq.org/
* Ruby: for creating Selenium-Ruby based test scripts
-- http://www.ruby-lang.org/en/
* Rake: Build programme
-- http://rake.rubyforge.org/
* NMQA Vienna: as a (free) test management tool
-- www.freetestmanagementtool.com
* MS Virtual PC: To run various OSs to use with Grid
-- http://snipurl.com/pij0i
* SciTE: Text editor
-- http://www.scintilla.org/SciTE.html
* SQL Server Management Studio Free Express edition for working with SQL DBs
-- http://snipurl.com/pij7

So, if you have to rebuild you laptop tomorrow what technologies and tools would you be putting back on there? If you went to a client site tomorrow to deliver some testing what would you expect to be working on?

Mark.

9000 hours of billing with no single bug found yet

2009-07-31T16:02:00.000-07:00

In a discussion on Test Republic at http://www.testrepublic.com/forum/topics/code-coverage-with-test-cases Pradeep Soundararajan summarised the evil, immoral, corrupt dark heart of the commercial side of the testing profession so well I wanted to make sure his words were captured here so they wont be lost. The commercial reality is just that, a reality that we can’t escape but I hope we can generate amazing revenue while adding value and avoiding fleecing our clients. Too many consultancies have been sued because they didn’t – you know who you are.

Reply by Pradeep Soundararajan on July 28, 2009 at 6:17pm
Mark,

Thanks for not posting it earlier. It gave me an opportunity to get you post it.

I do exercises in my exploratory testing workshop that demonstrates that those who seem to care so much about RBT aren't actually caring about it. As Michael Bolton pointed out somewhere in Test Republic that those who profess so much about documentation themselves don't bother to read and write good documents.

A good RBT requires skills that people appear to be reluctant to build. By saying that I don't mean, "Yeah, I have built it". I have been trying to develop it as much as possible and constantly practicing it so that I am prepared for a war anytime.

My perhaps more militant stand against RBT (the misuse of terms and vocabulary for ‘coverage’ aside) is because the Behaviour Driven Testing approach I advocate more and more these days focuses on behaviour relevant to the user. Ticking off Requirements against test cases is of no use to the user. Answering the questions above and formulating testing around them AND the requirements is. "

Exactly. You would call it misuse and businessmen would call it fair-use. I am starting to realize that the scripted approach survives because there is more money out there for business people through that.

Mark, outsource work to me:

I would spend a couple of days analyzing your requirement document and bill you for X hours per person involved in my team.

I would spend a couple of days writing a test plan document ( and not refer to it ) and bill you for 2X hours per person involved in my team for preparing it.

I would spend a month or two writing test case document ( and refer only to it ) and bill you for 10X hours per person involved in my team for preparing it.

I would then again create a traceability matrix ( just to fool you and your boss about our coverage ) and bill you for 5X hours per person involved in my team for preparing it.

18X hours per person involved in the team. Assuming X is 50 hours and there are about 10 members in my team, that's 18 * 50 * 10 = 9000 hours of billing with no single bug found yet. If you are paying $20 an hour per person on an average, you would have actually given me a business of $1,80,000 without me or my team finding any bug yet.

Then comes the test case execution cycles and more billing. Why wouldn't a businessman be glad about the traditional approaches to test software?

Lets bother about the users of your product later during our maintenance billing phase ;-)

Lack of vendor support for Open Source

2009-07-28T12:41:00.000-07:00

The lack of vendor support is a real issue for Open Source and Free tools. It may seem logical that paid-for-tools are going to get superior support by the folks who have actually created them and have a commercial interest in promoting them.

This can certainly be true and NMQA (who I work for) have the Vienna test management tool that we created and so support both through paid service contracts and queries raised by the test community. However, the matter isn’t as simple as proprietary tools get superior support over Open Source or Free ones.

For example, NMQA also offer a Selenium-Ruby automation framework (in various forms) )that we support as aggressively as Vienna that we wrote fully ourselves. The reason being is that we see no difference between the two in terms of the support a customer needs. For NMQA there’s no difference in the support a customer needs for a proprietary solution we’ve developed and written in proprietary code or an Open Source / free framework constructed of open source code that we’ve set-up for them.

It’s when a customer tries to hit the internet and read online documents and forum postings to do it themselves the trouble starts. Think about that for a second, inexperienced staff trawling through spurious sources of information as the way to learn and implement a key technology, what a ridiculous strategy. Yet it’s the one often taken. Open source tools are not an easy solution to adopt unless there is expertise available, in-house or via a consultancy. The learning curve that inexperienced internal staff will take on is usually too great a burden for organisations to support and won’t deliver anywhere near as fast as is needed. Add to that the lack of trusted sources of information and we begin to see why organisations are shying away from Open Source.

There’s the issue – organisations trying to wing it on their own with Open Source solutions will mean they suffer more pain than if they buy proprietary tools and a service agreement. The best way is to engage a consultancy or specialist individual who can provide the same level of support you’d get buying a support contract for a proprietary tool and that way there’s no difference between proprietary or Open Source solutions. Later on the difference is saving tens of thousands in service contracts as insurance in case something goes wrong, also ridiculous.

Mark Crowther.

Code Coverage with Test Cases?

2009-07-22T08:38:00.000-07:00

It hasn't really struck me until now - Why do testers think of coverage in terms of code - why aren't we thinking of coverage in terms of what the system does or should do? i.e Behaviour?

Thinking in terms of what the system should do is why we're testing isn't it? Isn't that what the customer / user wants us to be making sure before they get the software? Isn't focusing on behaviour how we assure that UAT is a success?

Never in my career have I ever really done 'code coverage'. I've played with it, talked about it with developers, even helped define accaptable coverage levels but I've never run 'code coverage tools' and declared my tests cover xx% of code. The developers I've worked with have, I remember this being the way when I was at EA. They put together Unit Tests and Component Integration Tests, ran the tools or did-the-math and declared coverage at a certain %.

What I have done however is declared coverage of Requirements. I've analysed what Test Scenarios exist, things I would do with the software to demonstrate the Requirements had been delivered on (coded the right thing), then Test Cases to exercise the Scenarios (coded the thing right) and find those lovely bugs.

Hmmm.....

It's testing with a focus on Behaviour

Mark.

What are your Selenium challenges?

2009-07-22T08:34:00.000-07:00

It seems that when using Selenium natively there are a number of common challenges people encounter. Here’s my list of things I encountered and thought “Hmm... how do I do that then?”
What would you add? What have you struggled with or are struggling with now?

• Dealing with pop-up windows
• Testing dynamic text or content
• How to go about testing Flash
• Capturing screen shots, either to file or in some form of report
• Iteration of the test case, running repeatedly with minor changes
• Data Driven Testing, pre-cooked data or generating on the fly
• Generating useful test status reports
• Setting up Remote Control
• Setting up Grid

Mark.

Recording bugs in development, don’t bother.

2009-06-24T16:05:00.000-07:00

There was an interesting thread of tweets on twitter (no less) started by TestingGeek I believe and asking about raising of bugs in agile projects.

In a traditional / heavyweight projects I guess it’s a done deal. We find bugs in the testing phase, actually any phase post the development phase, and they go through the bug lifecycle of log/triage/assign/fix/re-assign/re-test/loop-de-loop/close. No worries about tracking the bugs, building bug taxonomies, doing root cause analysis and corrective action planning and of course generating those all important measures and metrics.

I think I first heard Elisabeth Hendrickson suggest that bugs in an agile projects should just be fixed and not logged/tracked. On first hearing this it seemed to be a heretical suggestion. What about all that lovely stuff I can do when tracking bugs for one and aren’t I letting Developers ‘get away with it’ for another? Hmm.... Then I recalled that when I was at CAPS-Solutions working for Martyn Arbon about 4 years ago he’d suggested roughly the same thing. Test stuff early and get the Developers to fix it while they’re still on the project. The last person to subscribe to this perspective was Tien Hua at NMQA.

It struck me today that I’ve never really considered what my view is on this. In recent years I’ve definitely been letting the Developers get away with it. I distinctly remember a project at CAPS that went out earlier than previous projects, with more in, with more testing done and of a quality that surprised the customers. Full life cycle testing too right up to compatibility and UAT and we finished about 2 days early if I recall correctly. Problem was we recorded next to no bugs but we were finding them, that I do know.

What I remember that felt new was we were talking to the Developers as we found bugs, encountered issues, got stuck. Essentially the test team were paired with a Developer, reporting in issues encountered to the Developer as they were found. A quick fix and retest later and we were off testing new cases. Testing progress was fast, test cases were moving to a passed state and the relationship with Development was positive. I hadn’t heard of Scrum, agile, et al at that time.

These days I’m inclined to follow roughly the same approach but any that are non-trivial are recorded. Trivial being the Developer can literally change it there and then as it was a brain glitch moment (variable not initialised, variable scope context, incorrect typing, wrong table or file name referenced, etc.). Sure we might need a rebuild to get the change but the change is done in the time it takes to shout across the desk. For non-trivial bugs found at this point I’ve (once) had the team post a Bug Card on the wall so it becomes part of the backlog of tasks to be worked on that sprint/iteration, a blue sticker get’s attached when it’s accepted/estimated/assigned, orange when fixed, green when closed. I recall some getting accepted and de-prioritised and resolved in later releases.

If the Developer is still coding and progressing that bit of code towards being ‘done’ and ready for integration - the more I can do to help that and not be a tester ‘getting in the way of progress’ the more I feel I’m fulfilling my role in the team. On a personal level I’m not interested anymore in catching-out the Developer when they’re still working on something that’s being crafted. It isn’t fair and doesn’t help. I wouldn’t want the same being done to me when I was writing tests but insight that helped me complete them would be/is very welcome. If we think about this for a moment it’s easy to see why Developers have got so peeved with testers in the past.

The change comes when the Developer declares they’re ‘done’ and (usually) moves onto the next item to code up, having integrated the current item so the test team can run the next stages of testing. Now there’s a slight distance between the Developer and the code they wrote starting to grow. Just like in a traditional approach where a Developer might complete development for one project and move onto the next. From here on, from Integration to the end of the product’s life sat in production/live I see the value of applying a bug life cycle that then supports any tracking, taxonomies, RCA, etc. that might be found useful.

What’s your views on this? Do we ‘miss out’ by not tracking ALL bugs? Where should tracking not be done/be done?
How have you balanced the desire to measure/manage v make progress/collaborate?

Elizabeth Hendrickson gets it, Tien Hua gets it, I think I get it, how about you folks?

Feedback to PM's - weekly or daily?

2009-06-22T13:05:00.000-07:00

Driving on my way to work today I started thinking about ‘feedback’ on projects and how often we’ll give feedback to a Project Manager on a traditional project. My experience has been that the feedback is generally given once per week, at the Monday morning Project Management meeting.

I’d almost forgotten that this had been the way I used to provide feedback on how testing was going. Feedback once per week would see me having kittens now. I’m so used to the agile style of running a daily stand-up that the thought of weekly updates just seems so alien.

I imagined going to my online banking service and after logging in seeing a message that said ‘No updates, please come back Monday’! A moment after thinking this I realised in this situation I would immediately panic and be ringing the bank. A realisation which came a moment before I realised I’d missed my turn off! Doh!

June SIGIST Talk - 'Pragmatic Testing - the Middle Way'

2009-06-18T07:17:00.000-07:00

The following text is the transcript of the talk given by Mark Crowther at the June SIGIST in London.

What is Agile/Heavyweight/Testing anyway?
Often we see on forums, blogs and so on discussions that ask about and responses that aim to provide a definition of what Agile testing is in practical terms.

In the same way we might use say Waterfall as a definition of what Heavyweight is, it sometimes seems that we're looking for a diagram or model as a way to define Agile.

Or perhaps by saying that Heavyweight is all about documents and process we can compare and say Agile must therefore be about avoiding documents and process.

The issue is these approaches of simple comparison will always fail to help us clearly define Agile and Heavyweight. The fact is that right now Agile and Heavyweight are at best metaphors or perspectives on how to approach testing practice.

They're not clearly defined paradigms, i.e. there's no robust and complete definition of the practices that make up Heavyweight or Agile test approaches. So in my view we as a profession don't clearly and collectively know what being in the Agile testing mode of practice is but we expect and are expected to deliver as if we do.

We don't have a clear set of practices and approaches that we can point to and say 'that's agile or that's heavyweight'. We have a few random things that sit in each area but mainly a subjective idea of what it means to be Heavyweight or Agile.

Who's read or contributed to discussions on forums where the question is about the tester’s role in an agile team?

It comes up every 4 or 5 weeks on the forums. The other one being 'I been told we're going agile, what does that mean'?!?! You can almost hear the tester’s frightened voice behind the question.

The very fact we're at a SIGIST dedicated to testing 'in practice' tells us there's still a lot of thinking and discussion to do, not just around Agile either. If it was all signed, sealed and delivered we'd be talking about something else, here and within the testing community generally.

I previously worked at AOL and when I was there I had the good fortune to work with Thoughtworks. If you're familiar with them you'll know they could readily be considered thought and practice leaders in the agile development domain.

In fact around 8 years ago, on a project they were in to help deliver, they introduced us to crazy new ideas such as Daily Stand-ups, backlog items and Wikis. They were doing agile development and testing, 8 years ago... and here we are today as a profession still discussing what it means to do Agile testing in practice.

I believe there's a fundamental reason for this and it's that our current way of thinking about what Heavyweight and Agile testing practice is - is deeply flawed.

It's flawed to such a degree that if we continue in the current mindset we'll carry on re-asking the same questions for years and still not arrive at the answers we're looking for.

False Expectations of the Testing Future
My first contention is that we hold a false expectation that the testing profession is undergoing an evolution from a Heavyweight view of the testing world to a more Agile centric one.

In the same way we might consider ourselves to have matured from an ad-hoc / unstructured testing world into the heavyweight.

That in some way we'll become progressively more enlightened about how to realise testing process and practice that's moves us more towards Agile and in so doing moves us beyond the Heavyweight.

It's my view that this destination doesn't exist and that in fact we're already seeing what the future of the testing profession will look like. Consider what happens in practice when you're on projects and planning or actually delivering testing.

You may consider a project to be an essentially heavyweight one, perhaps it's testing in a regulated industry. But how often in those projects are you asked to create all the documents you proposed;
- but could you maybe not write out the test cases with all the steps?

Perhaps the project is lightweight requireing a more agile approach but how often are you then asked to work from iteration to iteration...
- but provide details and durations of all your tasks for the entire project just for budgeting and planning purposes?

- or could you provide a more complete Gannt chart as that's what the customer is expecting to see, instead of just the dashboards you were planning to provide?

Even in Ken Schwaber's book, Agile Project Management with Scrum, he mentions at the end of almost each chapter that he has to step away from 'pure' Scrum and compromise on the way he would like run things, in the ways just described.

The evidence from my experience and what I've heard and read from others is that the reality of testing practice is neither purely traditional/Heavyweight or Lightweight/Agile, but is more of a Hybrid of the two.

I'd suggest that the future of testing practice isn't going to be defined in terms of Heavyweight or Agile as we think about those perspectives at this time, but in the future testing will defined in ways more similar to the Hybrid approach that we're experiencing on most projects today.

I believe that this hybrid approach, driven by the project's constraints and practicalities of the delivery, is more representative of the 'normal' state of testing practice that we experience.

However, while we continue not to realise that 'hybrid is actually normal testing' we'll continue to hold the mistaken belief that we must be purely heavyweight, agile or it's not quite right.

What's more, in so doing we'll continue to try and evolve into Agile from heavyweight at the exclusion of practices and techniques we keep experiencing the need for.

Meaning and Impact of a Testing Paradigm
Adopting this perspective of hybrid testing as normal, we can more easily move towards defining a central, stable paradigm that more accurately describes the core of our profession.

In saying this I define a paradigm; as being 'a set of exemplary practices that define the core principles of a discipline'. This collection of practices come together to give us a definition of normal testing in the same way we have 'normal science'.

Normal science is often referred to as 'thinking inside the box' and represents the day-to-day accepted ways of approaching a scientific discipline.

The idea of a Normal Testing paradigm can be considered in the same way, as representing the exemplary set of practices we'd use in the day to day testing situations we'd expect to find ourselves in.

I'm not talking about defining immutable best practices here by the way, but I do believe it would be acceptable to refer to them as good practices. In many professions this set of practices is collated into a peer reviewed Body of Knowledge (BoK) by perhaps the governing, chartered institute of that profession.

We have the start of that in the ISTQB but it doesn't go far enough as the ISTQB doesn't publish the actual knowledge, just the syllabi for various courses.

Defining the testing knowledge relies instead on accredited commercial organisations designing courses or authors writing books that provide material that can be used to deliver the syllabi.

Not everyone follows ISTQB of course, as such the testing profession's approach to defining the actual knowledge that would deliver our 'normal testing paradigm' is at best commercially biased, at worse fragmented and un-coordinated.

This situation therefore perpetuates the issue of us never being able to 'finally' define what Agile, Heavyweight or a normal testing paradigm might look like.
At least not in a way that is consistent, agreed and accessible to everyone in the profession.

It's this situation that I suggest causes us to remain confused about topics we knew about 8 years ago or to ask the same questions on forums every 4 weeks.

It's also what causes us to perceive the work of testing luminaries, such as James Bach, Elizabeth Hendrickson, Michael Bolton, etc. as the new, emerging paradigm we must follow or get left behind.

It's also what causes us to feel exasperated as to what we should be learning and the approaches we should be taking, and why there seems to be so much disagreement about what should now be; done-and-dusted fundamentals.

It's attempting to make sense of this is why I say that as most of us go through our careers we inevitably sway from Heavyweight to Agile to somewhere else and often feel very confused along the way.

It’s also a barrier for new comers to the profession and limits our standing as a serious bona fide profession in the eyes of the rest of the world.

So what to do given the current situation?

The Middle Way – a Toolbox for Testing
About 12 months or so ago NMQA had a series of Workshops in-house that essentially touched on the points I've made so far.

We realised that even when called in to work on what were suggested to be Heavyweight or Agile projects they were always – ‘Agile, but...’ and ‘Heavyweight, but...’ or something entirely different.

Now as a consultancy we’d responded to these situations and delivered - but drawing on our previous experiences as managers and members of test teams we recognised getting in this situation isn’t limited to just consultancies.

During our careers as test professionals we’re going to encounter differing environments, maybe we’ll work in;
- digital media or the online space where a more ‘agile’ approach is needed
- pharma or military where regulation needs a more ‘heavyweight’ approach

This is the ‘normal testing paradigm’ that’s at the heart of our profession and being entirely focused, schooled in, beguiled by the Agile or Heavyweight perspective is going to limit us.

The practical outcome of this perspective at NMQA was to develop a ‘Testing Toolbox’ that contains all of the core models, practices, processes, documents, etc. that we would reasonably expect to need for most projects.

We used this to define in real terms, objectively the - ‘normal testing paradigm’ as NMQA experience it, a collation of everything stable, accepted, we felt could place ‘inside the box’. This is the same idea as I mentioned for the scientific paradigm earlier on.

But what about testing practices and the work of people such as James Bach, Elizabeth Hendrickson, James Lyndsay, Matt Heusser, Lisa Crispin, Michael Bolton, and many others I could mention?

It’s important that they and the practices they promote, and agile in general, are not mistaken as the sum of the testing profession even if we often fall into talking about them as if they are.

Continuing the analogy from earlier a lot of their work I’d suggest are great examples of ‘thinking outside the box’. These people are thought leaders, practice leaders and they challenge and stretch us by what they think and do.

Like experimental science the work they do, pokes, prods, stresses, refines, re-defines, replaces, improves and tests the accepted body of knowledge. Every now and then from this we get a new or more powerful tool in our testing toolbox.

Exploratory testing is probably the most recent example that springs to mind.

We need to realise that 80% of ‘us’ need what makes up the normal testing paradigm as we have days jobs where we can’t experiment, we don’t have time or opportunity to dally with interesting and amusing experimental practices.

This is how I see these luminaries, these thought and practice leaders, as a vital part of the energy and vibrancy that the profession needs to overcome the inevitable entropy that would occur in maturing the profession in the way I’m discussing.

Getting Involved
There’s two things I don’t have time for in this talk today, Questions and the opportunity to show you the tools in the toolbox.

What’s more this is the first time this perspective has been presented to the testing community. We know what it looks like within NMQA but I’m interested in seeing what it means to the wider testing community.

Please do something for me - what I want to ask you to do is participate and contribute to this discussion. Here’s how;

• Visit www.softwaretestingclub.com and go to the forums. There’s already a post there waiting for your feedback, thoughts, disagreements, alternate perspectives.

• You’ll find a link in that post to a survey about this talk, click the link and complete the survey.

In return - if this talk has got you thinking, if the ideas of moving beyond the ‘agile v heavyweight’ discussion, if the ‘normal testing paradigm’ resonates with you and if building a ‘Testing Toolbox’ for your organisation is of interest...

Drop me an email or call me and I’ll come and visit you and your team and have a Workshop type meet for an hour or two where we can run through some of this again, I can show you some of the practices, template documents, test techniques, where we can think about what it means in the context of your organisation.

It’s my view that we need to fundamentally rethink the way we view our profession. The old perspectives have served us well so far but I don’t see them doing so well in the future.

I’ve enjoyed talking to you this morning and I hope this has been of interest to you.

Thank you.

Mark Crowther

Thoughts after June BCS SIGIST

2009-06-18T02:18:00.000-07:00

I enjoyed SIGIST this time, last time I went I had the overwhelming urge to scream, something like "What are you people thinking!?" or maybe just ""STFU!" which may have ruined what little reputation I have as a thoughtful individual. Thankfully Michael Bolton was there this time so the bar was raised.

I was also seriously pleased to have a few beers with him and other folks last night and today just spend time absorbing what he had to say. James Lyndsay was there today too, I think he sneaked (snuck?) in. It was a bit of a double-take when I saw him having never met him before either. My colleague Ian has, he went on James' Rapid Test Course and is a changed man. Michael suggested a line of study to me, it's little things like this that mean a lot.

It's not wrong to say that the thinking Ian is coming up with now combined with my own stuff is rocking our clients world. 18 months ago when I joined NMQA I finally got time to really 'think' (write, post, ask, rethink) and study the work of folks like James B, James L and Michael (crap loads still to learn of course). In a few months after being taught by James L my colleague and I are as good as on a par with each other regards the really kick-ass test approaches.(Yes we still ahve different experience, unique perspectives, etc. we're not twins)

I think that having been absorbed in this way of thinking is why I'm writing this blog when i should be in bed. It was the presentation after me that has me twitching, I couldn't f*&^%&ng believe it frankly. I doubt he'll ever read this blog hence my uncharacteristic spleen venting. Dood, are you stuck in a f*&^%&ng timewarp? Listen to me.... "No, no, no, no,no" to infinity.

I can't remember the last time I wrote a 'real' Test Strategy in the sense of the 47 page monster example I have on my laptop that I copied when I left a previous employment about 7 years ago. I remember liberating it because I thought it would come in handy, it didn't. I just checked my website too and see I have a Test Strategy Template there.

I have a confession to make, after posting it up there maybe two years or so ago I've never used it. I hate them, I don't write them, I contribute to a Project Strategy if there's one. If I'm creating document like this I write Test Plans that are never over two pages.

Strategies like this are as bad as the typical test case suites that make me want to slit my wrists because it would be quicker than watching my life being wasted with them. If I intended to spend 2/3 of my time writing documents, updating them, explaining them, version controlling them and have become a bloody documentation clerk or administrator. I wasn't hired because I can use word.

By the way, I really don't care if I never see a formally reviewed, 40 page, version 2.0, signed and sealed Specification. It won't stop me testing, I don't need it to test, in a twisted way I have more fun when I don't see them. Anyway, it makes my cringe when the organisation I'm working in churns these out.

As for 'why am I testing if there are no requirements'? There shouldn't even be any code! So it won't be a problem, even if the document is missing I've got this crazy ass idea, I could talk to the developer! (Conference, Reference, Inference - Lessons Learned)

We're testers, so ffs focus on testing. If you want to look after all that crap become a project manager or something. It kills me to see what you presented, no one cares, just focus on testing.

At NMQA we have a couple of phrases that get reused often:
* "Just enough definition, just enough control - nothing superfluous to needs" and
* "Test stuff - a lot!"

Test, test, test, that's why we're here, be militant about testing, the doing of it. Not being able to test because you're waiting for documents is an excuse when there's software available, an excuse incompetent testers use.

Go look at how you can use Bug Reports as test cases, post it notes on a white board as a requirements catalogue, shake your thinking up and read, read, read, read, read, then go read Lesson Learned in Software Testing.

It's hard, it's uncomforatble, it's work to change but the change in perspective that's possible will blow you away and make testing the most exciting, challenging, engaging, fulfilling job you can stay up late and blog about!

I feel better now, time for bed.

Behaviour Driven Development

2009-05-13T04:54:00.001-07:00

Behaviour Driven Development (BDD) - The future of testing.

My view is the BDD is the bridge between development and testing, representing the paradigm shift in thinking that's needed to ensure closer integration between the development and testing professions. BDD provides a logical interlink between what I define as 'Test Requirements', which are drawn from functional requirements, and Test Cases executed to prove these Test Requirements have been achieved.

The BDD approach allows the Test Requirements to more naturally and completely emerge during development. The tester can then relate functional requirements to implemented behaviour and write effective tests cases that demonstrate the customer has what they wanted in a way they wanted. Using solutions such as Selenium/RSpec a tester can create automated tests that help move functionality more rapidly to a ‘done’ state and keep in the spirit of focusing on behaviour.

Shifting thinking from TDD to BDD also eliminates the (curious) mental block development can suffer around testing they do and testing testers do. The BDD mindset re-informs testers as to their valuable role within teams using the approach and eliminates the (also curious) identity crises testers can suffer when being part of an agile team.

Why there are bugs in software

2009-04-26T02:39:00.000-07:00

Why bugs are present in software?

The development of software is a complex discipline the output of which cannot be exactly the same as any previous time or approached in exactly the same way using exactly the same techniques.

It’s obvious the exact actions that are used to develop software cannot be repeated precisely. This is not manufacturing, there are no poke-yoke, failsafe software development system, fixed-gauges to ensure precise quality is achieved or robotic systems to reduce human input.

The development of software involves large amounts of human involvement, ambiguity, ambiguous statements of requirements, a myriad of possible solutions and uncountable potential failures.

This means two things are guaranteed:
• Bugs will be found at every stage of the SDLC.
• Every application released will always have bugs present within it.

That's my thought for the day lay here on the sofa like a cat in the sun, lush green sunny Spring days, nice.

The Irrational Tester

2009-04-21T06:59:00.000-07:00

Just read a great paper by James Lyndsay of Workroom Productions.

In it he talks about the various types of irrational bias that we can suffer from as testers. Read the paper here.

Confirmation bias = Test Cases?
Process Imperialist, Agile Evangelists = Congruence Bias?
Clustering Illusion = Bugs are where bugs are? How to avoid false clusters?

Under Illusion of Control james states "When testing... we seek reproducible experiments". So, testing is an experimental activity? James Bach thinks it's a science. Testing is an Experimental Science then?

In the same section he discounts testers (unlike traders) being "vi) Goal focused". testers aren't goal focused and so this isn't of Illusion Control? That doesn't seem right unless I've not understood (likely). Aren't testers very goal focused?

Broken Windows, paragraph 4. Yes, it will lead people to become sloppy because that's what happens. people are a) lazy or b) suffering some form of bias mentioned in the paper.

Mark.

Wordle Analysis of my Blog

2009-04-19T16:13:00.000-07:00

Slightly off-topic but I just ran Wordle against my blog here and this was the result:

http://www.wordle.net/gallery/wrdl/762115/Cyreath_Blogspot_Wordle_analysis

title="Wordle: Cyreath Blogspot Wordle analysis"> src="http://www.wordle.net/thumb/wrdl/762115/Cyreath_Blogspot_Wordle_analysis"
alt="Wordle: Cyreath Blogspot Wordle analysis"
style="padding:4px;border:1px solid #ddd">

How can China become the world leader in Software Testing?

2009-04-09T14:25:00.000-07:00

There’s two aspects to this, firstly there’s the individual test professionals and secondly the testing companies and organisations in China.

Assuming there's an active testing profession in China, lot's of skilled, well educated testers who are all doing great work and assuring the quality of world beating software, then the first step is for these skilled testers to say 'hello' to the rest of the world, just like you're all doing by being part of Software Testing Club

I could give you a list 20 people long of test professionals that I would say are the most prominent or active in the worldwide testing community - but they'd be British, American, Australian, European and Indian. Not one would be Chinese. There must be Chinese software testing forums but what / where are they? People from all of the above countries/regions can be found on this site as well as on other websites, so where are all our Chinese friends talking about software testing? Why aren’t we seeing them on English language forums?

At this point we hit possibly the major issue, English is the lingua franca of the testing profession in the above countries/regions. So if there’s to be wider global collaboration for the profession it’s going to have to be done in English. I know that’s a bit one sided but that’s the reality, the world isn’t going to learn Mandarin.

That means testers in China need to speak and write English if they’re to fully interact with the countries/regions that already actively collaborate. I know many China based offshore development centres and large consultancies already have this as part of their business approach (E.g. CSC, Bleum, Microsoft, IBM). It’s an obvious thing to do if these companies want to work outside of the local Chinese market with their international customers and partners.

So far I’ve suggested individual Chinese test professionals need to become more visible to the global software testing community, they become more collaborative with the worldwide testing profession, learning and sharing with a sense of equality and shared purpose, utilising English as the language of the global profession.

There’s one further aspect to China becoming the leading light in software testing and that’s the companies and organisations involved in software testing.

I’ve said before that I’ve contacted organisations in China, and continue to do so, to discuss how to collaborate with them. The response has been stunningly poor. Examples include proposing the writing of a test training course that could be used freely by the Chinese organisation, I never heard back from them. Another the offer to an individual of a collaboration on Papers and Podcasts, again nothing happened.

I’ve asked you guys here and on other forums about who leads the testing profession, who’s been on CSTQB courses, authors of Chinese software testing books, magazines, trade shows like the SIGIST meetings we have here, etc. and everything I’ve been told is on the China Testing Club here. It’s about 5 lines of material. The Chinese software testing profession seems invisible, almost insular.

The worst response to date was the most recent, where I offered to run my Test Practice training course for free when I visit family in Shanghai in September “honestly speaking, hires professional testers, so there is fat chance to have the training; also we have our in-house trainer”. Wow, so much for collaboration with the global testing community. That put me in my place! ;]

What’s the steps in summary then? Encourage Chinese testing professionals to;
• Get involved with the global testing community. Signing up here is s great first step.
• Write papers and essays on software testing and share them here and other forms, collaborate with non-Chinese test professionals such as me.
• Regularly write a blog here so we can see what they’re thinking and get insight into testing in China.
• Develop relationships that allow sharing of forums, papers, podcasts, etc. across websites.
• Pair up for cross-mentoring between Chinese and non-Chinese test professionals.

My final thought is that a Chinese test professional making the effort to become known and collaborate in this way can easily make themselves known globally as a notable figure within the Chinese testing community.

Mark.

Legacy Industrial Strength Test Software

2009-03-18T08:13:00.000-07:00

QTP & QC.

That's what prompted the phrase above in discussion with my colleague, the idea that these tools were 'industrial' scale heavyweight tools and due to the amount of time they've been around they are getting legacy. In my experience I still hear about these tools (obviously) but on the ground see them less and less.

Many times I get called in to help organisation with using the tools - because they bought them and use say the Test Case management module or no longer use QTP because the automation guy left.

What I'm seeing is more and more interest in lightweight, open source tools for test management and automation. Selenium, Watir, FIT/FitNesse being the usual candidates.

So the question is are the "Legacy Industrial Strength Test Software" tools days numbered? What's the next 1, 2, 3 years going to look like for tools?

Teaching the ‘Product Life Cycle’ in my Test Practice course?

2009-03-11T04:53:00.000-07:00

Why I teach the ‘Product Life Cycle’ in my Test Practice training course.

For those that have attended the Test Practice Course* you may recall we open up by introducing the Product Life Cycle. Most folks in software development and testing start thinking from the Development Life Cycle onwards. So, why do we teach about the Product Life Cycle?

From the course you’ll know that we put the Software Development and Testing Life Cycles in context with each other and the product Life Cycle. In this way we learn how the Test Life Cycle supports the Development Life Cycle which in turn supports the Product Life Cycle. What we wait to discuss later in the Test Management Course is how awareness of the Product Life Cycle allows more effective management of the test function. How so?

When a product is released from the development phase and goes into maintenance (live) then the project is usually considered to be over. The test and development teams are done with it and they move onto the next project. This is a fallacy for most organisations, because it simply doesn’t work this way. More often than not the reality is developers and testers are dragged back to fix and test issues with live products. Some organisations are a little more aware of this need and have a process of bug-bash days for just this type of activity. The problem here is it’s approached as something that needs brute force to address and make go-away.

The Test Manager who wants a more complete understanding of how effective the test function is will be keen to know what happens when a product goes live. Thinking back to the Product Life Cycle the Test Manager will be mindful of how long the life span of product is in the market. They will consider the Product Life Cycle Phases and take interest in measures such as; No. Of bugs found overtime, across Life Cycle Phases. How many of us know that? How many bugs, by severity per functional area get reported by your customers?

With this insight a Test Manager can assess needed testing resource for the duration of the products life, analyse what bugs are found when and start to effect a quality improvement plan, this allows assessment of effort and costs and a way to show reductions in the cost of quality and reduction in cost of ownership in real terms.

That’s two reasons why we teach the ‘Product Life Cycle’ in our Test Practice training course.

.....................................................
Head over to Software Testing Club to book your place:
Read about the course here
Use the enrolment forms to book your place

Move Test Teams to Agile?

2009-01-11T16:37:00.000-08:00

In the LinkedIn group "Senior Testing Professionals" a question was asked about how to move testers who are used to Heavyweight approaches to a more agile test approach.

It rather depends on how agile is being implemented in your organisation. Moving to agile is likely to mean an approach of either; Agile = SCRUM, agile = SCRUM + XP, agile = SCRUM + XP + TDD. How is it happening in your organisation? Process and Technical changes? Have you already begun, if so what’s the current situation?

Obviously this is another topic we could write a book on so I had a think what would be my top three changes the team would have to take on board.

Irrespective of what agile means as described above, here’s three key considerations I’d suggest for moving a team that’s been delivering in a heavyweight development environment to a lightweight (agile) one.

Firstly they need to adjust from focusing on the bulk up-front analysis and test case authoring they may be used to. Remember the documents they’ve been expecting may simply not be ready up-front of each Sprint / development phase. They need to plan, analyse and design based on what they know now and in a way that’s flexible enough to accommodate what’s to come.

Secondly, test execution has to be effective. Effective means finding bugs early and quickly then proving stability near the end of the development phase. This may mean they need to learn new approaches such as Exploratory and be much stronger on analytical techniques, domain knowledge, test case design, bug reporting and near-cause analysis. There’s no room for tic-toc testing.

Thirdly, they must maintain and develop frequent high-bandwidth communication, relationships with the other project team members and focus on interaction and not communication by proxy (documents).

Scratching the surface, but I've seen the above ignored and it's painful to see!

China - Software Testing Club

2008-11-08T06:01:00.000-08:00

Over at http://www.softwaretestingclub.com/ I just set a club up for China.

I have a special interest in China (having family there) and in Software Testing and I have a problem - finding out about Software Testing in China is like haystacks and needles.

At a Client site recently I encountered a number of Chinese linguists doing translation work for software. It occurred to me I've never encountered a Chinese Software Tester in the UK. Indian sure, Chinese, they don't exist it seems.

I know off hand of two companies that are involved in offshoring test and development to China, CSC and Bleum. I know a tonne of them in India.

I know of Indian testing qualifications, universities that teach software testing to some level, I know a few Indian testing luminaries too. I have no idea about courses, conferences, institutions or guru's (chánshī) in China. (Slight lie, I know of this training provider but it's in Chinese so am working off dodgy Google translations http://www.beidaqingniao.org/)

Is ISTQB established in China yet? I've contacted the person marketing it over there, we'll see what response I get.

Test Estimation - Research and Thoughts

2008-10-17T09:16:00.000-07:00

Test Estimation, what is it and how to do it.

I think this easily joins the list of "Top 10 Most Asked Questions" that would be put together by trawling any popular forum or testing community website. There's an idea...

Someone being able to estimate, a) How long it will take to write Test Cases and b) How long it will take to execute them; is a fundamental for any test team. This assumes the person doing the estimation is reasonable at the test analysis activities or has someone in the team who is, a Test Analyst maybe?

Estimation has a number of factors of course; complexity and scope of the testing needed and of the item under test, number and competency of testing staff, the impact of development on the testing effort or more accurately the impact on the quality of the test item, the list goes on.

In Q1 2008 I had a sit down with myself and drew together a collection of thinking I'd done around this topic. I'd hoped to formulate a model that was sophisticated enough to be of use but not so convoluted that it was unusable. I've seen a lot of vague discussion about estimation and a lot of nice but complex theory that would never be usable in the heat of a project.

One usable approach can be seen in Tony's Video and there's even a worksheet on his website.

From my side I did write a paper on Test Estimation though it contains several approaches and not one - I just knew it would the moment I started writing. That I believe reflects the main issue of why we keep talking about this topic, there's no one-size-fits-all and I suspect that the 'answer' is more likely a set of guidelines and approaches, not axioms or rules.

Apply these in context of your environment to make them relevant and augment them with additional guidelines that make your approach work for you, in a way that you understand.

My paper and a JavaScript/HTML paper is posted
on the site here. It contains an image of how to complete the HTML page too as it may not be overly clear.

One other approach is to do a decomposition of the work to be done. During test analysis we'll usually try and identify all of the individual peices of functionality to be tested. Creating a collection of scenarios or test objectives for each of these and then defning a set of Test Cases. From here we can work out the individual timings for writing the Test Cases and then Executing them. In total we'll have our estimate for test prep and running.

Here's a copy of a Test Estimation Workbook that could be used for this approach. Perhaps using this after the 'main' estimate is done might be interesting. I wonder if we'd have the time!

This approach means we can also track planned against actual effort to provide some clever Measures & Metrics reporting and use the data to feedback into our future estimates.

Mark 'roughly this big' Crowther

20th UK Test Management Forum

2008-09-29T19:23:00.000-07:00

I just noticed details for the next UK:TMF have been posted:

http://uktmf.com/index.php?q=node/127

This is the free and informal meeting that happens each quarter. Well worth a visit to enjoy a conference that's run in a way that encourages open debate and knowledge sharing.

Hope to see some of you there.

Mark.

Communication Training or your life

2008-09-22T22:38:00.000-07:00

MSB Executive http://www.msbexecutive.com/

I've just had one of the hardest days of training, heck one of the hardest days, I've ever had.

NMQA kindly set me up with Martyn "I will rebuild you" Barmby of MSB Executive for a day of Communication Training.

It started gently enough with some discussion about how I approach presentations, why I dislike sales (activity, not people) and moved into everything from how I stand and walk, enter a room, make small talk and greet people.

Later on we covered the SPIN technique (Situation, Problems, Implications, Needs) which can be used to structure presentation of products and services. Then came the role play, walking about while speaking to control breath and delivery, followed by a 'therapy session' on any mental blockers I might carry about with me that impede my performance.

This type of training is so hard because in order to benefit the most you need to set-aside what you think and know and think you know. You need to take on board every comment, engage in activity that makes you feel uncomfortable. It drains you physically and emotionally.

Has it helped?

Following the training I had a Client meeting where they opened with "You probably can't help us.." and ended with them agreeing product presentations, collaboration on developing testing standards for thier industry and sending a pilot job to us as a trial. Yes, it helped!

The next meeting with another Client I proposed a spend on testing equal to 10% of total annual revenue. Hey, testing's important dammit ;] I gave the best presentation of my life, addressed all needs and managed all issues. They agreed to the proposal and we're now two people on site.

I admit it, Martyn has taught me how I can enjoy Presentations AND Sales, wonders never cease.

Yes, the training works. This is the website you need:

http://www.msbexecutive.com/

Mark "I should have had this training 10 years ago" Crowther.

The Cost of Quality

2008-07-03T17:09:00.000-07:00

Read the Cost of Quality Discussion Paper

One of the key areas of Quality Management that I try to promote is the understanding of quality costs to the business. It's all too easy to know there are problems with the business operations, be those Software Test processes or Software Development, but never to really understand how much they affect the bottom line.

The paper provides a worked example based on public information available for EA Inc. As stated in the paper this applies to every business and the principles discussed can be applied to SMEs. In fact there are two good examples that I use time and again as to why SMEs must understand the Cost of Quality principles:

It's critical that SMEs see maximum benefit from often rarefied financial resources. They must remediate money wasting practices now and avoid wastage scenarios in the future. Addressing the Cost of Quality can provide both the remediation activities and provide an avoidance strategy.

An SME that is experiencing serious growth or is holding a strategic piece of the market may be a candidate for acquisition. They should understand the Cost of Quality to ensure investors are shown the management have these costs under control.

The actual experience however is that a business won't embark on a Cost of Quality programme, why? Many reasons such as fear of the unknown, can they face up to knowing and do they have the drive to complete the programme? If the business appears to manage costs enough to make profit they may feel a lack of motivation, not a world class attitude but very common.

What's discussed in the paper is a big step for any business to take. The rewards are great but the work is long and hard. However, the trick is in adopting the Cost of Quality mentality from day one. Then a programme never starts as something separate, it's just a natural part of the business practice as are reaping the benefits and avoiding the future issues.

Mark Crowther, Manager of Costs

QA Survey, Interview Questions, Awareness Campaign?

2008-05-13T13:00:00.000-07:00

Some time ago myself and a colleague were asked to brainstorm a list of questions that might be used in an internal survey. The purpose of which was to assess how aware the organisation was of what the test team did.

It was one of those great 1 hour brainstorms that produced a peice of work that surprised both of us.

Having been searching my PC for something entirely different it was a pleasant surprise to rediscover it. So, before I lose it again I thought it might be wise to share it ;)

Enjoy using these and add any in the comments that you think of yourself.
..................

Reporting
1) Do you know what type of reporting QA provide during a development project?
2) Are you aware of any reporting or information that is available when QA are not engaged in a development project?
3) Do you know if QA publish any information such as what they are working on or who is working on it?
4) Are you aware of what could be provided using any reporting systems we might have?
5) Are you aware of any reporting systems QA might have?
6) Have you ever seen reporting for areas such as Bug Quantity found, % complete of the test cycle, % code coverage?
7) Are the above or similar of use to you? How?

Testing
8) Is there a process for requesting testing / test resource from QA?
9) Do you know what type of testing QA provide during a development project?
10) Are you aware of the testing that QA perform when not engaged in a development project?
11) What reasons do you know of for QA selecting a type of testing to deliver?
12) Can you name any dependencies QA have from other teams that help them deliver the testing?
13) How do these dependencies impact QA delivering the testing?
14) Is it clear why testing takes the time it does?
15) When a Test Engineer isn’t testing, do you know what they are doing?
16) Is it clear how early QA should be involved in a project to deliver effective testing?
17) Do you know if and how QA use Requirements and Design documents?
18) Do you know what items or documents QA create when delivering testing?

Bugs and Issues
19) When do QA submit bugs?
20) Is it clear why QA will submit bugs?
21) Against the above do you know of any reason when we wouldn’t submit a bug?
22) Is the process for managing bugs through QA and dev clear?
23) Do you know the process for deferring bugs?
24) And closing them?

QA and Testing Knowledge Share
25) Has it been made clear what the ‘Software Test Life Cycle’ is?
26) Are you aware of any other practices and processes QA have in place?
27) Is so, do you know where they are published, if at all?
28) Do you know what tools QA use to aid testing?
29) Do you know if QA, development and documentation use the same tools?
30) Do you know how the outputs from tools get used by different teams?
31) Do you know how many Test Engineers are in the team?
32) Are you aware of how they are assigned test work?
33) Do you know if QA have relationships with development teams?
34) If QA do, is it clear why?
35) If QA don’t, do you believe QA should?

Contentious…
36) Have you ever attempted to build a relationship with the QA team?
37) If you have, how did this go?
38) What would you suggest is the single most useful thing QA do?
39) What would you suggest is the single most useless thing QA do?
40) Do you believe QA add value to a project?
41) If not, why not?
42) If so, how?
43) Can you name a project that would be in a lesser state of quality without the input of QA?
44) Can you think of a project where QA didn’t help or perhaps were detrimental to the project?
45) If QA could make one change to what we do, what in your opinion should it be?
46) Do you think it’s clear to QA what your team do?
47) Come to think of it, what is it you do? :]
48) Do you believe QA should be proactive in chasing development or development should come to QA?
49) Do you see QA as a potential pool of resource for adding to development, a stepping stone into development?
50) Should QA provide guidance to the developments teams on how to test their own software?

Test Case execution priority

2008-04-16T00:26:00.000-07:00

A question put to me recently:

"If you have Test Cases already prepared and you have to just excute those test cases in a very short time then how will you decide which test cases has to be excuted first. Is there any technique ?" - Mrityunjay

If you have more Test Cases than you have time to run then you clearly need to put them in some priority order to ensure you run the 'Most Important Tests'.

I think it's rare that the test team can do this as we don't have a complete view of the importance of all development features that are the Most Important.

Approach 1

As soon as you have them written get together with people that can decide their priority such as the Architect, Product Owner or Project Manager and Walkthrough the Test Cases. Ask them to put them in order for you. Ask them "If I test nothing else, if I show nothing else works - what would you want?". Then when you have that ask them "If that was done, what would be next?" until you're done. This meeting should be 30 minutes tops.

Approach 2

If the test team really have to choose then there's analysis needed to decide the selection of the cases:

Risk - Does the Test Cases cover an area where there's been a lot of change or where many issues are usually found?
Importance - Does the Test Case cover an area that absolutely must work and/or will always be used?
Breadth - Does the Test Case cover functionality broadly and is a high level test over a set of lowere level, alternate Test Cases?

If you have a Test Case that is covering critical functionality in a high risk area then you need to run it. If you have a Test Case that is covering an area of minor change across functionality hidden several menus deep that will be used by a small group, leave it to the last.

Of course now you need to make sure you're reasonably accurate in your estimation of how long test execution will take. This will ensure you can declare what test will be executed and which wont be.

Test Estimation, take a step back

2008-03-27T03:28:00.000-07:00

As test professionals we're expected to answer 'off the cuff' how long something will take to test. It must be the single most common dreaded moment for any test lead or manager. The projects starting and we're expected to know ho long all test tasks will take, including the identification, authoring and execution of tests. Sadly, 'How long is a piece of string...' doesn't go down well nor does sarcasm about how it's pointless estimating because after the project owner/customer/developers delay/change/fail to write documents, etc. the estimate will be worth poop. As true as it might be.

So, these 'challenges' aside how can the test professional at least arrive at a reasonable set of estimations for the test tasks? Firstly we need to know what tasks are needed. It's seems right that we should be able to break down a delivery of testing into a set of known or expected tasks. Having those identified we could then estimate how long each would take.

Taking a step back - it's essential that the test manager is clear on what tasks should be performed for a given project. That comes through designing a test methodology or several which can be applied to various project types.

A further step is to model the effort needed to deliver each task. For example, writing the Test Plan. Though it's impossible to estimate with perfect accuracy the test manager cannot make this up every time. They must have 'model' answers that can be refined in context of the project.
When I get folks who are convinced they can't possibly say how long I ask: 'How long does it take to write a test plan?' - 1 month? 1 hour? Well no. 1 week then? Too much. With all reviews, maybe 1 week start to end. The effort for this? By the test manager maybe 2 days tops. Great, so my model answer is a Test Plan takes 1 week to deliver and requires 2 days of effort by the test manager.

Test Case identification and authoring is more difficult as the complexity and number of cases needed is entirely context dependent. There's a greater reliance here on the project being run in a way that delvers such artefacts as good requirements documents. Ah yes, back to that old chestnut.

Next time I get a project I'll think about it's complexity, the competency of staff, etc. and provide a duration/effort in context of the project. But I have my model answer as a start point so I can arrive at reasonable estimation more quickly. Rinse and repeat for every other task in your methodology and you have a general estimation model(s).

Oh monkies, a Survey!

2008-01-01T11:38:00.000-08:00

If you're part of the QA & QC Google Group you'll have noticed my posting about analysis techniques and practices I put up there some time ago. Actually, you probably didn't notice as no one replied. But it's OK, I'm not (that) bitter. To help folks I've put together an anonymous survey. Non, no, don't thank me.

A few months ago I read Lesson 32 (chapter 2 generally) and section I-8 of the NRC Fault Tree Handbook and I felt I was getting a real insight into analytical techniques. You know those things you do naturally to some degree, but you do them in a crude fashion and you're not actually sure if they have a name? That was me, heck still is!

Well here's the shimmy, I've been studying the topic of testing requirements analysis for a few months now. Part of what I'm trying to understand is what we 'really' do in this area. This really is one of those areas where we know what we 'should' do but rarely do or get the chance to do.

There's a link to the survey and pop-up on my website. The vocabulary I use in the survey is ambiguous on purpose as I'm trying not to 'lead' people in their answers. I'm no professional survey author so hopefully it wont confuse you too much! It’s only 10 questions but you’ll see you can add as much extra detail as you wish.

Drop me a mail with your critique of the survey too. I'm very keen to see how these can be used as a tool to get more insight on topics of interest.

Remember the survey is anonymous so I can’t thank you individually, you can always email me of course! So, thanks in advance and I'll make sure I share the information here once I have something meaningful together.

Mark "and just another 50 questions" Crowther

The joy of YouTube

2007-12-28T03:04:00.000-08:00

I feel it's taken me a while to realise that YouTube is in fact a great resource for study of software testing. Using the Google Group I am part of or Wikipedia and listening to software testing podcasts are a given. But for some reason I hadn't sat down and spent time on YouTube, curious.

Well, that's the great thing about Christmas breaks. Time to eat too much, watch TV and spend time in front of YouTube.

If you're reading the blog looking at getting into testing then have a watch of the Portnov School's six video set that introduces the course they offer. They're in the US but the overview of what basics a new tester should master are the same the world over. I've encountered a fair amount of people that might have ISEB or similar but don't know some of the basics they seem to cover on this course.

A good video by the well renowned James Bach and is called Becoming a Software Testing Expert. Many comments about James have him viewed as arrogant and self promoting. To a degree I think he is but to think that's all would be to miss the essential message he tries to convey. That being there are no 'experts', there are no 'best practices' and maybe no spoons too. Keep this in mind when watching and it's a great hour with this interesting guy.

Another interesting video is on Agile Testing by Elisabeth Hendrickson. She starts by discussing the more traditional test approaches to put Agile in context. The main discussion picks up at around 13.00 minutes onwards. During the video she discusses Agile and XP and how they sit together, which was a useful perspective, in addition to how Scrum fits in.

Elisabeth mentions a number of things that aren't quite complete 'right'. One thing being the independence of the testing function. Stating it's not correct to have the function independent of the Agile team. In fact Agile teams support that independence through the equality of the function within the team.

As part of the Google Tech Talks series there's Scrum at al with Ken Schwaber which is a great follow up to the previous video. He's surprising in that he doesn't really evangelise the use of Scrum despite being the main creator of it. To follow up Jeff Sutherland then has a talk about Scrum implementation.

The interesting thing from both the Agile and Scrum videos is that both talk about the need for documentation. Something that is often assumed can be dropped as a mark of going Agile...

There's five hours of viewing there alone and many, many more videos to watch. So enough blogging, time for a cuppa and some more videos.

The Dreaded Walkthroughs and Reviews

2007-11-20T03:11:00.000-08:00

It's always been a source of bafflement (technical term) that within the testing domain Walkthroughs and Peer Reviews are not more widely practiced. I recall being 'invited' to a code review where a developer went through their code line by line in dolby prologic monotone. It was almost as painful as the many walkthroughs of Test Plans I've subjected folks to.

What I took away from the meeting was how incredibly useful and interesting it 'could' have been. Here was the opportunity to have code explained line by line, to be provided a live narration of the thinking, logic and reasoning behind what had been created. What's more our own views and opinions could be incorporated into what he been made.

If this was some contempory artist or author giving a narration of one of thier works we'd be fascinated and consider ourselves fortunate that we might influence the work. But it's just some bloke we work with and it's code, so we fall asleep.

The problem is it can be hard to get excited about code, even harder to get excited about someone talking about it! The reality is most Walkthroughs and Code Reviews are brain numbingly boring.

You've probably heard of and been subjected to Walkthroughs and Code Reviews of various types, the idea that an author of something (code, plans, schedules, etc.) will sit down with an interested group and walk them through, line by line, clause by clause, page by page, explaining what was written and why. Occasionally asking for input, "all ok?", "Seem to make sense?" so that after say 30 minutes or maybe an hour everyone has found they're not interested anymore and are half asleep. Actually, probably asleep. It makes me feel tired just describing it!

Peer Code Reviews
Peer Code Reviews on the other hand are meant to be snappier, more active and energetic. Think more in terms of having produced a usable piece(s) of code, say a set of core functions or an interface onto those APIs your buddy wrote. Then with this small chunk in hand get it reviewed. The review is say 10 minutes long, you're doing a Walkthrough but it's a lively narrative.

To answer your question directly - Best Practices
- Small, manageable review items: not weeks of coding
- 5 or 10 minutes review time: not 1 or 2 hour brain numbing marathons
- Grab reviewers: don't book out meetings, keep it proactive (but don't break folks concentration if they're 'in the zone'!)
- Actively seek review from different colleagues: cross mentoring and knowledge sharing
- Keep responsive to new perspectives: review isn't just bug finding, it's learning tricks and tips too.

Peer Test Reviews
The interesting twist for us in the test domain is that we can apply this approach to our Test Artefacts. When we're in an Agile environment it serves us well to be working in the same spirit as our developer colleagues.

Why not get Peer Test Review of those 20 or so Test Cases you just put together for that module? Incrementally delivering Ready for Execution Test Cases is a great way to help the likes of Project Managers feel relaxed that we're making progress on our planning and prep.

Doing the same with Test Plans, Designs, Breakdowns or whatever other artefacts you produce is also a win. This lightweight approach achieves our objectives but stops us getting bogged down in heavyweight process.

Follow the above Best Practices and keep the event lively, if you really must book out a meeting to coordinate review with several people at the same time, that's OK. Just go a little overboard with your presentation. Print copies in colour if you can or let folks access it on their lap-tops to save trees. Use the projector to make viewing easier, create slides that are already noted up or can be 'written on', what ever keeps the energy levels up.

Mark Crowther
Peering at you!

The idea of applying Peer Code Review to the test domain came from: http://www.jaredrichardson.net/blog/2007/03/14#peer_reviews, thanks Jared.

The Future of Testing?

2007-10-03T15:42:00.000-07:00

A message over at QA & QC Group, well made my blood boil slightly. Here's the question(s) and response I gave. Grrr... etc.

"I am working in MNC in India, Recently one of my team mates (PM level) told me that the future of testing is no scope, and he gave reasons."

To answer your question in succinct form, the future for software testing has never been better, the future is brighter than perhaps we suppose, it's brighter than we can suppose. Why?

1) If any software boom is down first the Companies fire Testers.
Wrong. This assumes the situation as it is now or was in the past, as it is in this transitional state, but you're talking about what the future holds. So this statement is wrong. The ones at risk are development, probably more like Technical Authors or Support Desk. The value the business derives from testing is eminently improved the more difficult the economic situation becomes. The logic for this is because there is a need to maintain a share of a shrinking market where the differentiator would be price and quality, yes in the singular because the are so linked. Testers are far cheaper than developers generally and in safeguarding quality are essential. Safeguarding quality ensures the total cost of ownership is reduced meaning a well tested product is a better investment. A product with higher quality means a reduced need for Support Staff there by bringing a greater saving to the business.

2) Developers can do our job + Coding.
Wrong. A simple reflection on the Division of Labour, Specialisation of Work theories will tell you that there will at some point be enough work and enough need for focus of effort by individuals who are particularly experienced in testing, to need people who's specialisation is testing. Turn this around, can testers also code as well as developers? We would always say No to this, we understand their specialisation, experience, etc. lend themselves to being developers who can test, just as it does us being testers who can develop. But the two professions are not fully inclusive.

3) In future comfortable tools are comming.
Wrong. This has been spoken of for many years and even the best of the Record and Playback tools fail at encountering the simplest of issues. This is the same logic as for the robots running their AI cleaning my house and making me a cup of tea as I type... I still don't have one. Even if we accept the suggestion that these tools become so all powerful they are acting like a tester, who's going to configure, run, maintain, mature what they do? Is this person by definition not a tester?

4) No investments on the Testing more?
Wrong. The very act of investing in the above in a desire to eliminate the tester is by its nature investing in testing. If we accept that the paradigm of how we currently define 'tester' will shift then you'll not get rid of testers. Again, what will happen is the boundaries between tester and developer will blur. I maintain it's the developers who are at risk in many areas.

All of the above statements your team mate made are based on the current paradigm of what a tester is. They are looking at what they understand the tester of today to be and in doing so are making statements about testers that were essentially existing yesterday. Being in the profession we're aware that the days of just hitting keys and clicking the mouse are over, it has been over for all but the most junior members of a test team for perhaps more than 10 years!

Today's and tomorrows tester is a much more technically savvy professional. They can develop test harnesses, stubs and drivers written in and interacting with a variety of programmatic languages, author complex data sets, work in an integrated manner with Agile teams on a level that blurs the boundary between tester and developer, use highly complex tool sets testing across the many components of the global system architecture and much more.

Today's and tomorrows tester is a professionally educated, examined and accredited professional, including but beyond that of a general computer science degree and some courses that a developer may typically have and soon potentially a member of a Chartered Institute. Putting them at the same level as Architects, Lawyers or HR professionals.

The key hitting monkey of yester year now uses techniques grounded in complex statistical and analytical mathematics, cognitive psychology and some of the best scientific research covering everything from computer technology to human logic.

As I said, the future for software testing has never been better, the future is brighter than perhaps we suppose, it's brighter than we can suppose.

Mark Crowther

Time Spend Analysis

2007-10-01T13:09:00.000-07:00

I just posted a new paper on my website which reports the finding of a six week study in time spent by the QA team.

At the start of the study we agreed a set of Key Tasks such as 'Reading Test Documents' or 'Waiting on others', getting those involved in the study to agree the tasks and definition of them.
Then, for the next 6 weeks 6 staff recorded their activity every 15 minutes. At the end of each week the stats were collated and charted out to see where the department, team and individuals were spending their time.

The results were certainly interesting and gave some valuable insights, such as:

Team members in work for 7.5 hours a day don't test for that amount of time.
Test Managers are available for testing under half of their working day.

No, really. I bet these two insights are obvious to those who'd read this blog but on presentation of the data to the client there was slight disbelief. I had to show them the Daily Tracking Sheets to prove I wasn't making the figures up.

The study participants also kept an Issue Log to track non-bug issues and start to get a broader insight into what else was impacting their testing.

Have a look at the Papers section of the website for the paper and hit the Templates button for the 'Daily Tracking Sheet' and 'Issue Log' templates I mention.

Let me know your thoughts. Plenty of scope to make this study more scientific, what would you do or is this good enough for what we were looking to prove? Does it apply outside of the study group and make sense in your environments perhaps?

Mark Crowther, Head of QA Time Tracking

Configuring Test Environments

2007-09-09T20:38:00.000-07:00

Read the paper on Configuring Test Environments

"It works on my machine!", I'm guessing there are many test professionals who've heard this too many times for it to be funny. The problem we usually encounter though is that we wander over to the developers desk only to discover that ... it really does work on their machine.

The other version of this is 'the customers are finding bugs, how did you miss them?'. Now, having overcome your desire to beat the crap out of them for asking that ill informed question, you might run their test case and find a bug. At which point you'd be well advised to review the way you test cases are writen, the steps they include or functionality they cover. But what if you can't reproduce the bug?

Deep joy, it happpens, but why do these situations happen? Simply put there's something different. That something might be test data, the software version, hardware, the testing algorithm your steps present , who's to say. The main thing is you need to know what that something is.

The easiest way to do that is have as many elements affecting testing defined in advance of encountering issues. Easier said than done? Actually reasonably easily done, you just need a sensible and considered approach. The approach when I'm setting up a Test Function is outlined in my paper, Configuring Test Environments, an approach that brings many benefits.

It's abstracted from ITIL and has proven very valuable in helping to be assured about the status of the environment in which software is tested. Assuming there's good software configuration happening your walking in the park...

Mark Crowther - Optimus Configuratus

Cross Site Scripting and other merriment

2007-08-17T05:33:00.000-07:00

I got asked today about security testing and what tests I would recommend as a minimum should always be run. It was interesting to be asked as no one has bothered before!

I'm guessing like me you've heard of Cross Site Scripting (XSS). However, perhaps similarly to me you've never had a good play with it. Sure, all test teams will run a standard battery of UI validation checks, but security focused checks? I bet most companies that should don't insist the test teams do and I know from experience it's a luxury to be running these, especially where there's no automation in place.

I wasn't surprised to find entire sites dedicated just to this area of testing. There's a great site over at http://www.xssed.com/ for the study of this.

Just for fun I then went and had a look over some of the sites I do / have done work for and hey presto, issues.

The most common issues I was able to invoke was through testing with a bit of JavaScript, the Alert box XSS approach. It seemed one form field or another would do something untoward.

The Cheat Sheet I used was over at http://ha.ckers.org/xss.html and just dropping down to using “< pulled some interesting issues too. My second non-surprise, how common it seems, I'm such a cynic aren't I.

I'm not sure how vulnerable being able to pop these alert boxes makes the sites, I need to gen up on my JavaScript more. However, one thing I do know is SQL and just popping ' and 1=1-- in form fields was far too many error for comfort. Playing with a few SELECT statements after seeing table data and malforming URLs using JOINs was even more entertaining.

In the past I've been fairly much using a standard battery of UI validation entries but I'll be adding to the set over the coming weeks for sure!

Mark Crowther, Head of teh haxzor team

The Importance of Documentation

2007-07-01T09:19:00.000-07:00

Documents and other artefacts

The QA team can often be heard asking for documentation, much to the annoyance of those being asked. Those being asked are usually Developers or Project Managers of course. The documents being asked for are variously titled and might be Story Cards, Requirements, Functional Specifications, Technical Designs, Preliminary Specifications, a collection of these or something slightly different.

Either way, from the QA perspective they provide an overview of the requirements the development will try to deliver on. They'll describe what the intended design looks like and the developers are coding against and also explicitly stated requirements from which the implementation specifics can be gleaned. By explicit specifics I mean, for example, not just high level schema but details of table rows, columns, fields, mappings, allowable data, format of that data and so on.

Why? Because if requirements are explicit then the Test Cases we write are going to be specific, in which case they wont be general. See that not so subtle difference there? If requirements are not explicit or documents are incomplete or ambiguous then the tests we can write will be general in nature. For example a test of this type would be "The correct data should be output from the order capture system", you'll see data, it'll look correct, nothing bad will appear to have happened - the test will pass. They'll only prove the system doesn't do something we wouldn't want it to do. That is, we'll only test that the system does something that when observed could be generally assumed as correct.

Not really a very useful test though is it? Writing good Test Cases is a bit like writing SMART goals, you don't have to guess if it was achieved or not, you already know what the planned, predictable, intended outcome will look like. We'll be observing not testing.

However, if no one actually states what the requirement is - clearly, explicitly, unambiguously - we'll have no idea how to test that the intended outcomes of the development work really did produce the product that was wanted in a way it was wanted. We can never write SMART, useful tests. We'll just write a whole raft of superficial, not very useful tests that'll probably pass because the outcomes don't look wrong and be very unlikely to find important bugs.

Mark Crowther - QA Manager and Documentation Clerk

Thoughts on Recruiting

2007-06-12T07:07:00.000-07:00

I've just completed a cycle of recruitment and it's reaffirmed there are some people out there 'swinging the lead'. I remain convinced that 80% of candidates in our profession are just not that good, or interested, or are overstating their experience, haven't internalised their material or a combination of these and other elements.

So, how to make sure the recruitment process goes well? Here's my 2p worth.

First step here is to get yourself CVs from an agency you trust. Ensure they understand you expect pre-filtering, you expect them to understand the testing domain and with their knowledge pre-screen candidates they present. Refuse to work with body-shop agents and feel free to ask YOUR AGENT what experience they have of the testing domain. At the very least you want to be getting CVs that are good enough you'd be interested in them, not just interested in trashing them. Good enough being they are pre-screened and aligned to the needs of your role.

Secondly, do telephone interviews. I never used to but I've learnt that for the sake of 20 minutes I can screen out many other candidates that aren't quite good enough. Get them to discuss their experience, bring the CV 'to life' for you and touch on what they understand of the role. I'd suggest you can get a sense of the character of the person and a sense of if you believe their CV.

Thirdly, interview face to face with a colleague. I've never run tests by the way. My interview consists of the candidate re-running their walkthrough of the CV "for the benefit of my colleague", the difference can be very revealing second time around. As in highly inconsistent. Assuming they seem consistent I then bounce them around questions wise, as suggested above, ask about practice, process, theory, experience - then jump back to a comment they made 10 minutes ago and ask how they think that relates/affects/supports another comment they just made. Look for them bringing thoughts together that demonstrates they KNOW what they are talking about, that they really have lived the content of their CV.

Finally, make sure you go in having read the CV twice, highlight sections in different colours, write out some questions before hand and put together some form of Interview Pack that has updated Org charts, Job Spec, copy of the CV, etc. For ideas have a look at the templates section on the website for some of the Team Pack documents.

If you hire them and they are rubbish, pull them up straight away, find out why, be aggressive/direct but supportive. Make sure they know you consider them to be under performing and define expectations of performance. If they don't meet your expectations then cut them without hesitation, do not let it linger on in the hope they will improve.

Mark Crowther
Holder of the Magical Appraisal Dust

Books, Papers, Forums

2007-05-23T20:56:00.000-07:00

Reading, it's a fantastic thing. But how many people actually do it? Other than blog trawling I mean.

I ask because folks always seem surprised I have a number of Software Testing, QA and general Management books on my shelf. Around 30 at the last count. How many have I read? Maybe half cover to cover. I have also half read many papers and articles on the interweb too.

That's just the thing, so few people read and read. Read books, blogs, other web sites, papers, discussion documents, thesis, dissertations. When was the last time you went over to Google Groups or SQA Forums and did some reading and replying? Is it a worry to expose your possible ignorance? If you go to those forums you'll see I'm comfortable exposing myself to strangers.

I wonder if it's the same with buying technical books? The possibility of not understanding and making yourself feel stupid or the sense that, God it's boring. But there's a couple of good news items here.

Firstly, if most people in our profession aren't reading then it wont take much effort to read yourself above them. A few concerted reading sessions and you'll be steps ahead of most folks.

Secondly, even if you have a hard drive full of things you'll read later (or finish reading, along with a bookshelf of half or never opened books, well at least you're building a reference library for when you need them.

If you'd like to see some of my collection then wander over to the Books section of the website. I'll be adding more there as I write my reviews. That's to make sure I read them, well it's worth a try.

Mark Crowther the Librarian

A Documentation Model

2007-04-28T21:12:00.000-07:00

Read the Documentation Model discussion document.

What's the next most groan inducing thing you can say to developers and management after "Let's introduce a Quality Management System!"? How about "... with a really good set of formal documents!". Yep, developing a Documentation Model that's how to gain friends and influence people all right.

OK, so as quality oriented professionals we know a rational approach to the documents we use is a good idea. We also know that some external groups such as those troublesome (kidding, we love you) customers and certification bodies keep wanting to see our documentation.

It sure is good to show off a little by providing documents that are clearly part of an organised and well managed set. But where to begin?

Well, leaving aside dry talk of such things as documentation audits let's just focus on getting some documents in place? Don't worry. I've developed a way for the software test team to put in place a structured system for test documents, that's easy to understand and maintain.

It can even be scaled into a full Quality Documentation Management System with a little help from a few tools and willing participants. But that's another dry part we can discuss later.

A document about documents, all in three pages or less. Wonders never cease.

Mark Crowther, Head of QA and Documents about documents

Secruity Testing

2007-03-26T23:32:00.000-07:00

These last weeks I've been drawn into the study of Software Security Testing. By that i mean testing of security features, not the infrastructure.

As if building the web site and playing video games while trying to learn Python wasn't enough. It's been a busy few weeks while I'm 'in between jobs'.

It seems that the area of software security testing has yet to reach the level of maturity that Software Testing is beginning to enjoy. Not that software testing, even your basic black box functional testing, is as universally accepted as software development.

Soapboxing that software security should be seen as a separate area of delivery, that needs trained and experienced professionals to deliver on it sounds like the evangelical standpoint that was adopted for testing a few years ago.

Yet, reading around you discover there are some luminaries in the field that are pushing for this to change. The likes of Gary McGraw, that CTO of Cigital, stand out as Security testing's own Cem Kaner. Visiting his software security website over at http://www.cigital.com/ and listening to the Silver Bullet Security Podcasts reinforces that.

A sister company of Cigital is Fortify and they've developed an interesting idea to approach the actual delivery of software security testing. Combine that with the need for building security into the product right at the design phase.

Over at the Cyreath website I've written a discussion document around software security testing, read it here: http://www.cyreath.co.uk/papers/Cyreath_Testing_Software_Security.pdf then come back to the blog and share your thoughts.

Mark Crowther - Head of SWT (South West Trains...)

Q-bit anyone?

2007-03-05T22:47:00.000-08:00

If you're off to Q-Bit Test Expo on the 22nd March drop me a mail!

No Testhouse this year, I hear a rumour they're doing every other year. Vizuri will be there along witht their partners Q-Bit who they get training services from.

Chris Ambler, QA Director of EA and ex-Rugby player (judging by the ear) will be giving an expo of testing Video games. £10 says he tells us you can't really write Test Cases for games ;p If he get's his black ball out I'm going to storm the stage...

Please God make the presentations interesting this year, I fear for the testing n00bs and thier hopeful scriblings as they listen to product / company keyed presentations in the hope of true testing enlightenment.

Mail me (Mark Crowther) if you want enlightenment and confustication, don't listen to tool vendors!

OK, see you there!

Mark Crowther - Testing and QA Gnu

Developing the Team

2007-02-23T22:07:00.000-08:00

Read the Developing the Team paper

One of the things I do in any role is Team Mentoring which in part can involve setting up an appraisal system for the Test Team. That may seem unusual for a manager that focuses on delivering solutions around Software Test and Quality Assurance. I'd agree it is unusual but often insisted on especially where:

There's no system in place in the business or at least not one that's being followed to any great extent.
The business wants the performance of their shiny new Test Function managed effectively and measurably.

Setting up a new Test Function or hiring in a Test Team is an expensive and somewhat risky undertaking. Executive management must not only be assured that the Test Function delivers from a technical standpoint but from a personnel standpoint.

Good testing isn't enough, the business wants good teams made up of good corporate citizens. Are you living the brand, living proof of the corporate values, evangelical like in your delivery? The Test Team, especially a new Test Team, will be under incredible scrutiny and they may be 'seen' to perform in all aspects.

See, now we bring some of the pieces together I bet it doesn't seem unusual at all? Why not read the paper and share your thoughts here?

Mark Crowther, Head of QA and Team Flowering.

Notes for Emerging Leaders

2007-01-21T13:31:00.000-08:00

Read the Paper - Notes for Emerging Leaders

How many times have you learnt something that seems so simple and common sense, yet it's taken years to discover it? We've all been there, perhaps knowing something but having no name for it.

My good friend and Test Manager Stuart Crocker and I had a braniac conflag and decided to codify some of our key learnings.

Rest assured that this is in no way TEH answer (4 teh w1n) to your managerial education but it IS a good start! Let's face it, you'd be an idiot to ignore our wisdom, you're not an idiot are you?

Enjoy and comment you slaktard.

Mark Crowther - Test Manager and Head of Queue Hay

Cross Team Process Synergy

2006-12-20T21:53:00.000-08:00

Read the Cross Team Process Synergy discussion paper

As Test Managers, we're often so busy delivering day to day that we never get around to properly thinking about how we could be working better. Sometimes we know we could be, we wish other teams would work with us more or that we had some form of working practice in place between teams - that meant there really was some form of team work going on.

The first step to making that happen is being able to easily articulate the benefits to teams, customers and the business.

Well I've collated those thoughts for you and even added some thoughts on how it'll create the conditions for quality. Once you've read this paper have a look at "A Documentation Model" and "Quality System Implementation", then come back here and tell me how it was for you.

Mark Crowther - Head of que aye?

Quality System Implementation

2006-11-20T13:59:00.000-08:00

One difficulty with doing something you've never done before is knowing what the thing you want to do 'looks like'.

It's the same with a Quality Management System. You can read all you like about the theory of it but sometimes seeing an example in front of you is by far the easiest way to really understand.

In keeping my approach of ensuring the fundamentals are made clear I've posted a paper discussing a small Quality System Implementation.

This wasn't for a software test team but a security team at a large Internet Service Provider. If you've read the paper "A Documentation Model" then this could be read as Part 2, a practical example. The discussion on scalability will make more sense after reading the paper here.

Mark Crowther, Head of Systematic Systemic Systems