Tuesday, 2 July 2013

A core set of regression tests

Software is complex and it isn’t possible to know the state of every class, method, function, accuracy of deployment, interaction with other systems to name a few things, at all times. When we have a piece of software that is improved over time, through the release of new and enhanced code, then the possibility we really don’t know if the code is good enough, increases exponentially.

That’s why we do regression testing, regression testing is testing that the code has not regressed, gone backwards, become worse than it was at a later stage. Regression testing is often ignored or paid lip service, do so at your own peril. I do understand why though, because it’s costly. It takes time to execute and people to do it and regression test packs need maintaining. So a minimum is done, just around the area of change, a quick look at key functionality.

Therein lays the risk. We take our eyes of the software and get to a point where we really don’t know what the level of quality is. Get that? Without a core set of regression tests, run every build -  we do not KNOW the software has not regressed, broke or become unstable. These days software is there to make money, unstable and broken software doesn’t make money, it costs money.

So how to make sure you have a good set of regression tests in place? A good heuristic is simply “What absolutely must work, can never be allowed to fail?”.

That’s your start point for deciding what needs a regression set around, then scale up from there. What live issues have been reported recently? Get some regression tests in place. Found some odd semi-reproducible defects during testing? Get the cases in the Regression test pack. Run the set each and every release, at least once, no excuses.

You may try and look good cutting testing time down, but your sure look stupid when the software breaks and you had a regression test you didn’t run. The customer is defiantly going to ask you (see what I did there..?) why you’re testing is out of control, when that live issue they complained about, that got fixed, has now reappeared.

A standing, useful, maintained set of regression tests should be a foundation stone to the testing regime, assuring quality and stability of your applications.

Thoughts? Send a message!


Liked this post?