Excel as a Test Management Tool

Because you're going to use it anyway...

Ruby-Selenium Webdriver

In under 10 Minutes

%w or %W? Secrets revealed!

Delimited Input discussed in depth.

Managing Knowledge Transfer Sessions

Read the article, grab the templates

Ask questions

If you don't ask, you won't learn. You won't teach either.

Friday, 4 November 2011

You don’t want to do ‘Penetration’ Testing, probably

Here’s an interesting thing, when people talk about Security testing they usually talk in terms of Penetration testing. You can imagine the scenario, “We need to make sure our site is secure, let’s hire a security consultant to do some penetration testing”. However, they rarely mean Penetration Testing. It’s the same as people talking about software quality assurance when they mean software testing. It’s similar to Performance testing when they mean Load or Stress testing.

It’s not that we don’t understand why people use these terms in a slightly inaccurate way, hey they’re not all 24/7 live and breathe testers like us, we can give them a break regarding the terms they use.

The point is, Penetration testing is more correctly thought of as; “A sub-type of Security testing focused on exploiting vulnerabilities present in the underlying design, application code and functionality and the network the application is deployed on. This includes exploiting interconnected software systems, networks, access controls and physical security.“ to quote the Test Hats ‘Software Testing Dictionary’.

Another good definition comes from the book, ‘Backtrack 4: Assuring Security by Penetration Testing’, “The basic difference between a vulnerability assessment and penetration testing is that vulnerability assessments identify flaws… without measuring impact, while penetration testing… exploits these vulnerabilities”.

Here’s the punch line that gets missed, the active exploitation of vulnerabilities should be expected to leave systems in a compromised state and that means they’ll need to be rebuilt. You did make a back-up of the data, configuration, etc. right?

That previous bit is the bit that doesn’t get mentioned very often it appears. No, not the need for back-ups… the bit about Penetration testing of live systems leaving them in a compromised state. Truth is, Penetration testing proper is a risky business and a service that most clients don’t really want and don’t need.

Most often what’s needed and what’s being asked for is an audit and assessment of how secure the network and application is. Of more value to them is seeing how robust their ‘security posture’ is in terms of technology, configuration, patch cycles and so on. They can then take a Vulnerability Assessment or Audit report and remediate any issues on there. This is far more valuable than Penetration testing on its own.

However/But… because there’s always a but… Penetration testing has its own unique value in that if you want to truly harden your systems, your need to do Penetration testing. The fact is that Compliance Audits, Patch Audits and Vulnerability Assessments are, in the main, checks. Audits are checks of compliance against a given standard, such as PCI-DSS or your own standards, they’re fairly static. Vulnerability Assessments are more dynamic, given the threat landscape changes day to day it isn’t possible to statically define some threats in an ‘audit file’ and merrily check them.

Penetration testing is not only more dynamic the testing approach required is far different too. This is testing, not checking or auditing. But… that’s a topic for another time!

Mark

Thursday, 3 November 2011

Best Practices - Smoke and Mirrors or Cognitive Fluency?

Simon Morely made reference to a blog post I wrote over at the Software Testing Club recently. It relates to Best Practices and a conversation we had in house here at Test Hats. He makes some great points and the post is well worth a read.

The Tester's Headache: Best Practices - Smoke and Mirrors or Cognitive Fluency?

The original blog post I made is linked to from his.

Mark.

Thursday, 13 October 2011

Why I added Lisa Crispin and Gojko Adzic to Wikipedia

Wikipedia is an encyclopaedic store of knowledge about the world, the people and events that have shaped it in the past and are shaping it now. The wonderful thing about Wikipedia is it contains knowledge about groups in the world, one group is the software testing and development profession/industry. This is relevant because software testing and development is at the heart of how the modern world is being shaped. The world and even those things we send out of this world all run on software. I want you to realise the significance of this and the importance of software testing before I gone on…

We are in the computer age and it’s as historic a moment for the world as the steam age and industrial revolution was. Yet, are we recognising this and ensuring we capture our knowledge of those that are shaping it?

Software Testing People

If you look at Wikipedia there is a category called “Software Testing People” (http://en.wikipedia.org/wiki/Category:Software_testing_people) and it’s a pathetic list of 15. (16 as I’m there, my user page is labeled to appear). We rightly see James Bach and Cem Kaner. I recognise Rex Black, Boris Beizer and Brian Marick. The rest? Never heard of them and I question whether they are really ‘testers’, but that might be showing my lack of ‘education’.

That aside, are you telling me that’s all the people who have or are causing paradigm shifts in the way we think about software testing? In how we approach it and re-shape the profession? Can you tell I’m offended and getting precious about my beloved profession? .

That’s why I’ve added Lisa Crispin and Gojko Adzic, I’ll be adding Michael Bolton too when I get time, please go ahead and beat me too it! These people have literally affected the entire profession. They have on their own and in combination with others (and the test community) caused a paradigm shift in the way we think about and do software testing (and from it, development). They have been around for years, are known and respected globally. Will be quoted, emulated and recognised (along with Bach and Kaner) as thought leaders and pioneers for as long as this profession exists.

You bet they should have Wikipedia pages!

http://en.wikipedia.org/wiki/Gojko_adzic
http://en.wikipedia.org/wiki/Lisa_Crispin

Wednesday, 5 October 2011

Testers coming to London

I get about 2 emails a week via the www.testhats.com site from testers looking to come to the UK, asking how best to do it. I also get emails about sitting ISTQB exams but that's another post.

I'm of the biased opinion the best place in the world for a testing career is London. If you are serious about a successful career in testing, then making the move there is a wise choice. One problem you will have though is there are many testers in London and some of them are very very good. Some of them are crap too, but you don't need to worry about those.

Therefore you need to do two things:

• Make it easy for potential recruiters to find you and learn about you.
• Differentiate yourself by becoming ‘known’ for your testing knowledge and views

In order to prepare yourself for arriving in the UK, you may have done this already but in case you've not here’s some things I’d suggest;

• Head over to http://www.softwaretestingclub.com/ and start interacting with the test community, it’s mainly UK based so will get you exposure
o Join relevant groups and interact: http://www.softwaretestingclub.com/group/uksoftwaretesters, http://www.softwaretestingclub.com/group/uklondon
o Set-up your blog there if you don’t have one, post at least once per week

• Join http://weekendtesting.com/ and the sister activity of Week Night Testing, then participate.

• Set-up your Linkedin profile - http://www.linkedin.com/ and get connecting to people
o Link-in with me: http://www.linkedin.com/profile/view?id=6436250 (Choose that you’re a member of STC)

• When in London be sure to attend the London Tester Gathering and meet the London test community
o http://www.linkedin.com/groups/London-Tester-Gathering-2656070

• Get a twitter account and follow other testers. Start to see what people are talking about and join in the conversation.

The most important thing you can do is participate in the testing community, either online or in person. The second most important thing is to start formulating a clear idea of your views on testing. For that you’ll need to study and practice, read and debate, then share your ideas.

You own your career and are to be congratulated on your determination to further yourself as a software tester. Please remember the correct order of things is Family > Work > Church and pay attention to them in that order.

Finally, I'd wish you good luck, but as I concur with Seneca who said "Luck is when preparation meets opportunity", I'll simply say "Get ready, then get on with it!"

Mark.

Tuesday, 30 August 2011

Security Testing Research, links galore

Over at the Software Testing Club I just added a list of resources for use by members of the Security Testing Group I set up. I thought I'd add the list here for reference and encourage readers to visit the STC group.

Websites and Forums

Dark Reading: http://www.darkreading.com/
Infosecurity: http://www.infosecurity-magazine.com/
Ethical Hacking Blog Site: http://www.ehacking.net/
The Ethical Hacker Network: http://www.ethicalhacker.net/


Podcasts and Video Series
Cigital Silver Bullet Security Podcast: http://www.cigital.com/silverbullet/


Security Testing Methodologies
OWASP: https://www.owasp.org/
OSSTM: http://www.isecom.org/osstmm/
ISSAF: http://www.oissg.org/issaf/


Threat & Incident Classification
WASC-TC: http://projects.webappsec.org/w/page/13246978/Threat%20Classification
WHID: http://projects.webappsec.org/w/page/13246995/Web-Hacking-Incident-Database
Taxonomy of Coding Errors: https://www.fortify.com/vulncat/en/vulncat/index.html


Tools

Backtrack: http://www.backtrack-linux.org/
NMap: http://nmap.org/
Nessus (Home Feed): http://www.tenable.com/products

Hack to learn, dont' learn to hack.

Tuesday, 9 August 2011

How to get started on SQL Injection

Firstly, you need a good working knowledge of SQL. That may seem obvious but you can't just rattle off a bunch of SQL strings and have no idea what they are meant to be doing, what they are testing for and expect to test well.

Head over to here and diligently complete each of the exercises:

http://www.sqlcourse.com/
http://www.sqlcourse2.com/


Secondly, get some pre-cooked SQL vectors to try out.

Go to http://ha.ckers.org/sqlinjection/ and try out the vectors MANUALLY


Do them manually to learn what they are, really read them and get familiar with SQL attack vectors. Try and construct some of your own given your knowledge of the app you're attacking.

Thirdly, Open Firefox and add 'SQL Inject Me'

https://addons.mozilla.org/en-US/firefox/addon/sql-inject-me/

Play with this add-on and see how it changes how you approach your testing. When you're done go to Firefox and click on "Tools > Add-Ons > Extensions > SQL Inject Me > Options > SQL Injection Strings" and add the bespoke vectors you created earlier.

Have fun!

Mark.

Principle Test Architect, Test Hats.

Friday, 4 March 2011

Security testing noobs and OWASP Breakers

I’ve said it before but one area of testing that functional/system type testers really neglect is Security Testing.

The neglect comes in two forms, one is thinking they know stuff when in fact they don’t. Most will rattle off SQL injection as the security test technique they know and it’s so ignorant and arrogant it makes my blood boil.

The second form is assuming security testing is the domain of rocket scientists types with PhD’s a l33t skillz in math. While true in some regards it isn’t the full story, just like SQL injection isn’t either.

If you’ve been in either camp I want you to stop before you do harm. The only way to do effective security test is the same way we do effective functional/system testing, diligent and consistent study and practice.

If like me the majority of your testing is against web sites and web applications then you need to head over to the OWASP website right away. That’s the ‘Open Web Application Security Project’ and the website has a host of free information and guides you can learn from.

Check out Michael Coates blog too, he’s trying to get a more active community going and one group is the Breaker group. As a tester go sign up and start to interact with the security testing community.

Get your skills honed and add real security testing to your arsenal of testing types.

By the way, Injection is number 1 on the OWASP Top 10 Application Security Risks list. So well done on getting that one. Number 2 is Cross Site Scripting (XSS), do you know how to do that? Here’s a vector for you:


Only 8 more to go!

Mark.

Monday, 28 February 2011

London Tester Gathering - May 2010 - Sign-up now!

I just looked over the growing list of talks and workshops being given at the London Tester Gathering in May and bloody-hell it's looking good. Not sure how Tony manages it but these gatherings just go from strength to strength.

http://skillsmatter.com/event/agile-testing/london-tester-gathering-2011

Given I have a lot to say about testing I fancied I'd request a slot to talk. But... I'm going selfish on this one as I want to be in the audience and learn from some of the folks doing the presenting and workshops.

There's already too much cool stuff happening across the two days to see it all!

If you do nothing else today go check the site out and sign-up. Book holiday, coerce your boss into giving you time to attend - just do what you have to do to be there!

http://skillsmatter.com/event/agile-testing/london-tester-gathering-2011

When you've read it please copy and send the Tweet below:

London Tester Gathering, May. Sign-up now! http://skillsmatter.com/event/agile-testing/london-tester-gathering-2011 @tonybruce77 #LTGMay RT!

Mark 'is it may yet' Crowther

Tuesday, 15 February 2011

Usability testing - a little checklist to use

Looking at the UI it wasn’t clear to me where the buttons were. That might sound odd but the graphics of the website were very ‘arty’ and it wasn’t immediately obvious where I was to click.

I remember another website I visited where I added items to the basket but wasn’t sure if they had been added. I then wondered if in fact the ‘add to basket’ button was inactive, maybe I needed to log-in first.

I’m sure we’ve all been on websites where the layout changes and our tour of it is suddenly halted while we work out how we use the site.

These are all examples of usability issues and though we as testers often know almost instinctively that this makes sites and applications ‘buggy’ but we might hesitate to raise them as bugs. The closest we come is probably functional testing with an uncertainty around whether these types of issues go beyond what we should be testing.

This is in part I think because as testers we don’t have a clear idea of how to classify Usability issues and in part because we might be stuck in a ‘requirements validation’ mind-set.

Getting out of just validating requirements is already being talked about a lot in the testing community so we can take that one as read. Having a clearer view of how to classify Usability issues we can talk about.

When I write Test Plans I have the usual system, functional test types but also Accessibility & Usability. I put the two together because many Accessibility considerations are affected by the Usability of the site.

When doing Accessibility testing we’re thinking about people with various levels of visual, physical and cognitive impairment that would benefit from a little ‘reasonable adjustment’ of the way we create software.

Usability is connected to this but extends beyond it, affecting every user of the website or application. Therefore this under-addressed form of testing needs to be factored into our test planning.

My guidance has come from the simple list below, courtesy of the Open University. When you experience a Usability issue in the software you’re testing, but you think it’s stretching the definition of a requirement – just label it as one of the below. Get the bug tracking software set up with a Usability classification to. Oh, and go chat with your company’s User Experience person and talk about this. One more team on our side :)


HCI Design Principles

Visibility in the context of UI design means making it clear what a UI element is used for.
Feedback is related to the concept of visibility. In the context of UI design it means making it clear what action has been achieved through the use of the UI element.
Affordance in the context of UI design means making it clear how a UI element should be used. Everyday examples include a cupboard handle to be pulled and a door plate to be pushed. When the affordance is wrong, a written sign is often needed.
Simplicity in the context of UI design means keeping things as simple as possible
Structure: A UI will be more usable if it is structured in a way that is meaningful and useful to the user.
Consistency in appearance, positioning and behaviour within the UI makes a system easy to learn and remember.
Tolerance refers to the ability of a UI to prevent errors if possible, or to make them easy to recover from, if not.

Source: The Open University, M150 Unit 12, ISBN0749257695

More information:
This book makes interesting reading: http://www.amazon.co.uk/Design-Everyday-Things-Donald-Norman/dp/0262640376
Website Design Considerations for Older People: http://www.usabilitynews.com/news/article2911.asp

Tools:
Web Accessibility Toolbar: http://www.visionaustralia.org.au/info.aspx?page=614
Colour Contrast Analyser: http://www.visionaustralia.org.au/info.aspx?page=628
Colour Blind Web Page Filter: http://colorfilter.wickline.org/
Colour Scheme Generator: http://wellstyled.com/tools/colorscheme2/index-en.html

Wednesday, 2 February 2011

Test Early and Often

On a recent project it was agreed that testing would mainly be executed at the UAT phase, I was to build a set of test cases that could be executed by the customer. While some testing might get done informally before UAT that was to be the main test phase. When UAT commenced there was delay while the test cases were proven and understood, before being run in anger.

Some test assumptions were wrong and test cases had to be updated. Some functionality wasn’t implemented as described in the requirements documents meaning more clarification and correction. Server issues further delayed the test progression and overall finishing UAT was delayed by around 2 weeks. Sign-off was achieved but it was done a little grudgingly based on contractual terms being met.

In an earlier project testing began before we even had the code. Test cases were used as a way to validate requirements and gaps were found. Not only in the thinking around the requirements but the third party applications that were part of the overall integration. Test cases revealed conditions that those applications wouldn’t be able to meet and a mitigation approach was agreed.

When new development was released tests were run straight away in part because automation was in place to assist. When UAT came around and the customer expected a final test run from us – we didn’t bother. There was no need for a final test run. We handed the code and test over and less than a week later got sign-off for the delivery.

These two very different approaches highlight the importance of testing often and testing early.

It’s not just about ironing out the issues early on. We want to do that of course. We want to find the issues in the requirements, in our test cases and assumptions, we want to have clear information in developer hands so they can code once and quickly. (Developers are artisans and artisans hate reworking what they’ve produced, they’re proud of their work like we are.)

Testing early and often saves time (and money), it cuts frustration and keeps people motivated as it gives feedback and insight, and confirmation of a job being done well.

Customer confidence and belief in the quality of work we produce, our ability as a competent and professional team all come across when we test early and often. In consultancy this is the holy grail of keeping customers and winning new ones. So many consultancies just do the job and follow the contract, it’s immoral, think of those that have been sued in recent years.

Build customer confidence by making quality and delivery a foregone conclusion by testing early and knowing it will be.

Mark

Liked this post?

Wednesday, 26 January 2011

You’re better than you think you are

Many software testers I’ve worked with, especially more junior staff, and not just to me as that’s getting easier every year, seriously underestimate themselves. One thing that’s helped me in my careers is an over developed sense that the work I produce is completely awesome. Accepting that to be true (see what I mean :p) it doesn’t mean that it’s ‘better’ than anyone else’s work.

James, Michael, Gojko, Pradeep, all do cracking work that’s impressive but some of what I’ve done matches it. It’s going to, they’re them and I’m me and we all have our own unique combination of skills, abilities and experience. That means I’ll come up with approaches and solutions that they could never think of, their own work speaks of a similar thing. Whether they have more public awareness of their work or not also doesn’t detract from how damned good my stuff is.

It’s the same for you too. You’re better than you think you are.

Everyone who has a passion for testing, is on the forums, chatting to other testers, reading or writing blogs, asking questions, going to conferences and gatherings and just ‘engaged’ intelligently and actively in the profession is as good as anyone else. By good I mean your ideas, thoughts, solutions, etc are as useful, insightful and ‘right’ as anyone else, known in the test community or not.

I want to encourage anyone in the profession who thinks their ‘ok’ at what they do to find the one thing they feel strong about. Maybe it’s writing test cases, using Exploratory testing, adding a splash of Ruby to the tools you use or some other thing you know. It doesn’t matter what- find that one thing and realise – you’re better than you think you are.

Take a deep breath, have courage and go and tell people about your one thing. Perhaps it’s a blog or forum post, maybe a 1 hour Skills Matter talk, a YouTube video or a paper/essay on your thing. People want to know and the test community is a very accepting and supportive community. They will share with you the fun and excitement of what you’re sharing with them.

It shouldn’t be a big event to speak at a conference, write a paper, present at a tester gathering. Get used to it, start now and watch how you grow as a person and a professional.

Everyone of us starts on the first step, sometimes we don’t feel too clever or have to admit we only really know our one thing. That’s fine, better to say that than try and act like a know it all. Some of my best learning and growth experiences have come about when I said ‘I don’t know’.

Too many junior testers are staying out of the limelight and you need to get in it. Sharath is a good example of someone doing generally good stuff and discovering he was better than he thought he was. Take his lead and get yourself out there, helping yourself and the test community at the same time!

Monday, 10 January 2011

What is Given-When-Then anyway?

I’ve been asked several times what a working set of Given-When-Then examples looks like in practice. The full style is not just the GWT component, though I have used that on its own. In full we might be having a Scenario followed by one or more Features with one or more Examples.

Here’s what a set might look like, based on some recent testing of Microsoft Office Communicator Group Chat. However, this could apply to any chat client or forum board.

You’ll see we have one Scenario that is the goal the business wants to achieve. To achieve it the software has to have a collection of Features (aka functionality) and to prove those Features are there we have Examples (of behaviour the system can enact – my definition of Functionality*)

Hopefully you’ll see how ‘obvious’ they are to any reader in any team due to the narrative format.
--
Scenario: Once the user has joined one of the Chat Rooms available to them they need to be able to read existing messages. These can include text, links, stories, files and images.

Feature: A user can see persisted and new messages in a Chat Room they have just joined
AS A user who has just joined a Chat Room
I WANT to see messages other participants are posting now and have posted in the last days
SO THAT I can read messages shared in previous days or being shared right now

Example: The user enters a Chat Room that has several days’ worth of messages
GIVEN the Chat Room has messages posted over the last days
WHEN I Join the Chat Room
THEN I can see the messages for the last days with a date stamp against them

--
Feature: A user can view attachments, links, images and stories contained within messages
AS A user viewing messages that contain more than simple text
I WANT to be able to open them and see the contents
SO THAT I can view the additional information and understand the message fully

Example: The user opens a message that contains a file attachment or story
GIVEN the message contains a file attachment or story
WHEN I click the message link to open the message
THEN the message opens and shows the attached file or story

Example: The user opens a message that contains a link
GIVEN the message contains a link
WHEN I click the message link to open the message
THEN the message opens and shows the link as an active hyperlink

Example: The user opens a message that contains an image
GIVEN the message contains an image attachment
WHEN I click the message link to open the message
AND hover my mouse of the image file posted in the message
THEN the image can be viewed as a thumbnail