Agile Testing Days - day 2

On the wednesday of Agile Testing Days it was my turn to speak, and together with Fredrik Wendt we presented "The Coding Dojo as a forum for teaching Test Driven Development". The talk was mostly aimed at developers, and the basic idea is to encourage them to learn TDD, and that going to a coding dojo might be a good way of doing that.

Michael Bolton
The first keynote of the day was by Michael Bolton, someone I've never met before. His talk really made me think. He presented himself as an "agile skeptic", and had some criticisms of agile practices like automated unit and functional testing. He talked about testing as something that humans do - it is a "sapient activity" requiring an engaged brain. He said "Automated acceptance tests do not answer questions about value" - that is whether the software will be valuable to the people who are going to use it. He preferred to call them "rejection checks" - that is they check whether the software will be rejected, but do not test whether it will be accepted. Only a human can do that. Michael went on to question the large level of investment that agile teams make in these kinds of tests. He thought that effort could perhaps be better invested, when studies show only 6-15% of problems are regression issues.

Michael emphasized the role of testers in projects as "skilled investigators" who provide a service giving you information about a software product. He drew on an analogy with a film critic - they don't tell you if the film has passed or failed, they tell you about the attributes of the film that will appeal more or less to certain audiences.

I liked the way Michael talked about testing as an investigative activity, and the role of testers as intelligent humans. I think he undervalues the feedback developers get from automated functional tests though. Agile teams do a lot more refactoring than other kinds of teams, and I think without automated tests they would have a lot more than 6-15% regression issues. I do think it is useful to evaluate investment in automated tests against other investments though. Some of the tools we have, particularly for functional/system level testing are not very cost effective to use.

I also think that testers can only start doing a "film critic" kind of role when the software is largely free of basic defects. These should be caught before a skilled tester gets their hands on the software, by developers properly checking their work, and automating those checks.