I begain using xUnit in 2000 and have been improving my skill at TDD ever since. Recently though I have been working less and less with xUnit tests, and more and more with using TextTest tests to drive development. This approach seems to preserve the original spirit of TDD, ie the tests are still driving development, but there are some differences and advantages.
When you use TextTest to drive development, your tests all end up being written in a representation that is separate from your code, ie a kind of Domain Specific Language (DSL). Your tests can be read and understood by anybody who understands the domain of the system under test, that is, they don't have to be able to read or write program code. You typically don't write new code when you write a new test, you just specify new desired program behaviour using this DSL. This makes it quick to add new tests, and easy to refactor your code later, since the test doesn't place any demands on the internal structure of your program (its classes and methods).
With xUnit tests, each test is only working with a small section of code, whereas a TextTest typically drives a path through a whole system or subsystem. This means that you need to use another technique to get fine grained visibility of what your code is doing. Instead of turning to a debugger, the approach with TextTest is to include log statements as you code. Having your application write a log is not a new idea and there are a number of excellent logging frameworks out there. TextTest allows you to not only specify the behaviour of your application in terms your customer will understand, but also control the application logging behaviour. When you make a code change, and your program no longer behaves the way the customer wants, (you introduce a bug), you can turn to the recorded application log to pinpoint exactly where program behaviour deviates from that previously recorded. So a test case consists basically of program input, expected user-readable output, and an application log.
TDD with xUnit is also generally accepted to be a design technique as well as a testing technique. I think there are two reasons for this. Firstly, just the fact of having these tests in some sense forces you to make good low level design decisions. Your code must exhibit loose coupling and high cohesion before you can extract units of code to test. This is desirable, and TextTest does not force you to do it, since it tests at a higher level of granularity. (On the other hand, if you are starting with poorly structured legacy code, where easily extractable units are hard to come by, you can get going with TextTest more or less immediately just by adding some (hopefully) harmless log statements. But I am not talking about legacy code in this article).
The second way in which TDD affects your design is that it forces you to think about the interface to your class before you define the implementation. This is low level design - choice of class and method names, and what arguments you give them. This is useful because thinking about the interface before the implementation should lead to a better, cleaner interface that is easier to use. TextTest doesn't help much with this either. It will force you to start by defining the interface to your whole program or subsystem, but it won't help on the class level. So when developing with TextTest we suggest something called “usage first” design. Start with the external interface to your program, and write code as if the classes and methods you want are already there. Then fill in the implementations, always working top down, and defining the usage of a class or method before your implement it. This takes some discipline, but leads to a similar effect to using xUnit.
A code Kata is an idea introduced by Dave (Pragmatic) Thomas. He points out that specialists in other disciplines do a lot more practicing than programmers do, in order to improve their skills. Others have further developed this concept into the idea of a “Coding dojo”, or a place where programmers meet to practice code Kata together. The website www.codingdojo.org is a resource for these communities, and amongst other things has a catalogue of various code Kata exercises various individuals and groups have worked on.
So that is all the theory, how does it work in practice? In this screencast, I demonstrate a PreparedKata of KataMinesweeper, using TextTest to drive development, in python. During development, I aim to work in a Test Driven manner, and I want to develop top down, usage first, in small steps. The idea is that although in real life you may not be so strictly test driven, or exclusively top down, or use such small steps, that it is important to know how to do these things, and they are worth practicing on a toy problem like this.
My aim is to demonstrate what it is like to do TDD with TextTest, and how you might solve this code Kata using the python programming language. If you would like to comment and give me constructive feedback on this screencast, I would love to hear from you.
The screencast is in four parts, and I estimate each part will take you about 10 – 15 minutes to view.
(Note: This screencast was created using Wink. In it I use Windows Vista Home Edition, Easy Eclipse for LAMP version 1.2.2 and TextTest version 3.9.1.)
At the XP2007 conference, Geoff and I presented a workshop entitled “The coder's dojo: Acceptance Test Driven Development in python”. (Geoff also presented the same workshop at agile2007). We had three aims with this workshop, the first was to use the meeting format of a coders dojo, the second was to do some coding in python and the third was to demonstrate how you can do Acceptance Test Driven Development using TextTest. We felt the workshop went well, we had around 30 participants and we were able to do a little of everything we had set out to do.
Perhaps the most important thing was what we learnt from the experience. The workshop participants gave us some very useful feedback. One thing people said was that there were too many new ideas presented to expect as much audience participation as we did, and that instead of trying to do a Randori style kata, we should have done it as a Prepared kata. There also seemed to be a view that the Kata we had chosen (KataTexasHoldEm) was quite a difficult one. Another very valuable piece of feedback was that we were doing Test Driven Development (TDD) with much larger steps than people were used to.
What I have done is to create a screencast in an attempt to address this feedback, and open up our workshop material to a larger audience. In it, I perform a Prepared Kata of KataMinesweeper, doing TDD with TextTest and python. Geoff and I have been developing this approach to testing for a few years now, and we think it deserves consideration by the wider agile testing community. There are important advantages and disadvantages compared to classic TDD with xUnit.
I will write an article about this approach and provide links to the screencast in my next post on this blog.
"One view, to paraphrase Kent Beck, is that manipulation is when you wouldn't getSomeone pointed out that they recently went on a training course where during one of the exercises, the course leader was not up front about its purpose. Afterwards he felt that this manipulation had led to a new insight that he was thankful for.
the same effect if you were upfront about the intervention."
(This quote is attributed to Michael Feathers)
Let me tell you a story about a young developer called “Sue”.
Sue has just learnt all about eXtreme Programming. She has worked on an XP team with lots of pair programming, unit testing, refactoring, simple design etc and is totally converted. The world should be doing XP!
Sue hears about another application at her company which is in trouble. Let's call the application “Squirrel”. Squirrel has been in production for a few years now, the codebase is rather large and the internal design is somewhat confused. The Squirrel team is having increasing difficulty adding the features the customer wants in a timely manner. Sue is convinced what they need to do is to refactor the codebase and add lots of automated tests.
Sue manages to persuade her manager “Paul” to let her work on refactoring Squirrel. She gets a copy of the code out of the version control system and sets about writing unit tests and refactoring. A few weeks go by and Sue reports to Paul that everything is going really well. She has solved some significant technical challenges in order to put in place some unit tests and has a small suite that she runs regularly. She proudly shows Paul the test scores and some UML diagrams of how the code looked before her refactoring, and after. Paul is pleased at how much better the UML looks and when Sue asks for more time to improve the code further, he grants her request.
A few more weeks pass and Sue is starting to attempt quite ambitious refactorings. She really feels the Squirrel team is going to be delighted when they see how much better their code looks and how many tests it has.
Meanwhile the Squirrel team has been continuing to add features and fix bugs and are actually quite proud of the progress they have made with the small team available to them. The users of the Squirrel application have complimented them on how much more stable and usable the application is becoming.
After a few months, Sue, Paul and the Squirel team agree that the next release of Squirrel should include Sue's refactorings. The approach chosen is for the Squirrel team to take Sue's copy of the codebase, and integrate all the recent bug fixes and new features into it. Everyone thinks this will be straightforward using the advanced diff&merge tools provided with the version control system. Sue persuades Paul that this would be better than applying all of Sue's refactorings to the Squirrel team's version of the codebase, since she had made so many sweeping changes and her codebase had extensive unit tests.
It was a total disaster. Sue had introduced a huge number of bugs when refactoring. Sue didn't understand what the code was really doing, having never actually worked on the application before and having only the documentation to go on. She had hardly spoken to the Squirrel team in all the months she had been working on their code, even though their project room was a short walk from her desk. The unit tests she had written covered only a fraction of the codebase and often reflected her poor understanding of what the code ought to be doing.
The diff between the two branches of the codebase was huge and the integration work was difficult, time consuming and error prone. After a week or so the work was stopped, Sue was in the doghouse, “Refactoring” was a dirty word, and the Squirrel team went back to their old codebase.
So what went wrong? Sue clearly did not have enough understanding of the codebase in question and was not skillful enough to pull off the refactorings she attempted. She had too much faith in the few unit tests she had written and lent on them too much. Sue didn't think about the integration cost rising exponentially the longer she worked apart from the Squirrel team.
However, I think it is Paul who really carries the blame for what happened. As the manager responsible for both Sue and the Squirrel team, he should not have allowed Sue to continue working alone for weeks on end producing only UML diagrams and unit test reports. Paul should have anticipated the integration problems even if Sue didn't. He should have asked to see working code – if he had sat down and manually tested Sue's copy of Squirrel, he would have found serious bugs within minutes.
Well, thankfully Sue wasn't sacked immediately and was given a chance to redeem herself. Sue joined the Squirrel team and started working on bug fixes and customer features. She quickly gained more understanding of the code than she had ever got working alone. Sue did put in place a few unit tests and do some refactoring, but it was always in the context of whatever bugfix or feature she was working on at the time. Most of the big sweeping refactorings that Sue thought would make a difference were never implemented.
The customer was happy for a while but the pace of new feature addition was still a bit slow for them. The customer also discovered that they could get a much better product from a competitor. So a couple of years later they dropped Squirrel. Happily for Sue, by then she had found a better job with a more competent manager.
class TesteAceitacaoCampoMinado(unittest.TestCase):
def testeCampo3x3Vazio(self):
campoMinado = CampoMinado([[0,0,0],[0,0,0],[0,0,0]])
campoMinado.preencheTabuleiro([['.','.','.'],['.','.','.'],['.','.','.']])
self.assertEquals([[0,0,0],[0,0,0],[0,0,0]], campoMinado.matriz)
def testeCampo3x3BombaNoMeio(self):
campoMinado = CampoMinado([[0,0,0],[0,0,0],[0,0,0]])
campoMinado.preencheTabuleiro([['.','.','.'],['.','*','.'],['.','.','.']])
self.assertEquals([[1,1,1],[1,'*',1],[1,1,1]], campoMinado.matriz)
def testeCampo2x3ComUmaBomba(self):
campoMinado = CampoMinado([[0,0,0],[0,0,0]])
campoMinado.preencheTabuleiro([['.','.','.'],['*','.','.']])
self.assertEquals([[1,1,0],['*',1,0]], campoMinado.matriz)
def testSmallEmptyGrid(self):
input = """\
3 3
...
...
...
"""
expected = """\
Field #1:
000
000
000
"""
self.failUnlessEqual(expected, minesweeper.count_mines(input.splitlines()).next())