Wednesday, 24 June 2009

Recording bugs in development, don’t bother.

There was an interesting thread of tweets on twitter (no less) started by TestingGeek I believe and asking about raising of bugs in agile projects.

In a traditional / heavyweight projects I guess it’s a done deal. We find bugs in the testing phase, actually any phase post the development phase, and they go through the bug lifecycle of log/triage/assign/fix/re-assign/re-test/loop-de-loop/close. No worries about tracking the bugs, building bug taxonomies, doing root cause analysis and corrective action planning and of course generating those all important measures and metrics.

I think I first heard Elisabeth Hendrickson suggest that bugs in an agile projects should just be fixed and not logged/tracked. On first hearing this it seemed to be a heretical suggestion. What about all that lovely stuff I can do when tracking bugs for one and aren’t I letting Developers ‘get away with it’ for another? Hmm.... Then I recalled that when I was at CAPS-Solutions working for Martyn Arbon about 4 years ago he’d suggested roughly the same thing. Test stuff early and get the Developers to fix it while they’re still on the project. The last person to subscribe to this perspective was Tien Hua at NMQA.

It struck me today that I’ve never really considered what my view is on this. In recent years I’ve definitely been letting the Developers get away with it. I distinctly remember a project at CAPS that went out earlier than previous projects, with more in, with more testing done and of a quality that surprised the customers. Full life cycle testing too right up to compatibility and UAT and we finished about 2 days early if I recall correctly. Problem was we recorded next to no bugs but we were finding them, that I do know.

What I remember that felt new was we were talking to the Developers as we found bugs, encountered issues, got stuck. Essentially the test team were paired with a Developer, reporting in issues encountered to the Developer as they were found. A quick fix and retest later and we were off testing new cases. Testing progress was fast, test cases were moving to a passed state and the relationship with Development was positive. I hadn’t heard of Scrum, agile, et al at that time.

These days I’m inclined to follow roughly the same approach but any that are non-trivial are recorded. Trivial being the Developer can literally change it there and then as it was a brain glitch moment (variable not initialised, variable scope context, incorrect typing, wrong table or file name referenced, etc.). Sure we might need a rebuild to get the change but the change is done in the time it takes to shout across the desk. For non-trivial bugs found at this point I’ve (once) had the team post a Bug Card on the wall so it becomes part of the backlog of tasks to be worked on that sprint/iteration, a blue sticker get’s attached when it’s accepted/estimated/assigned, orange when fixed, green when closed. I recall some getting accepted and de-prioritised and resolved in later releases.

If the Developer is still coding and progressing that bit of code towards being ‘done’ and ready for integration - the more I can do to help that and not be a tester ‘getting in the way of progress’ the more I feel I’m fulfilling my role in the team. On a personal level I’m not interested anymore in catching-out the Developer when they’re still working on something that’s being crafted. It isn’t fair and doesn’t help. I wouldn’t want the same being done to me when I was writing tests but insight that helped me complete them would be/is very welcome. If we think about this for a moment it’s easy to see why Developers have got so peeved with testers in the past.

The change comes when the Developer declares they’re ‘done’ and (usually) moves onto the next item to code up, having integrated the current item so the test team can run the next stages of testing. Now there’s a slight distance between the Developer and the code they wrote starting to grow. Just like in a traditional approach where a Developer might complete development for one project and move onto the next. From here on, from Integration to the end of the product’s life sat in production/live I see the value of applying a bug life cycle that then supports any tracking, taxonomies, RCA, etc. that might be found useful.

What’s your views on this? Do we ‘miss out’ by not tracking ALL bugs? Where should tracking not be done/be done?
How have you balanced the desire to measure/manage v make progress/collaborate?

Elizabeth Hendrickson gets it, Tien Hua gets it, I think I get it, how about you folks?