What to do next

I had fun for a few months programming Ruby at eLabs, but now I'm moving on*. What exactly I'll be doing next is not entirely clear. My plan, at least initially, is to work as an independent consultant specializing in automated testing and agile coaching. I'd also like to do some contract Ruby or Python programming. Hopefully I will find customers who are willing to hire me to do those things on a part-time basis, or with short term contracts, so I can do a mixture.

I have a long term dream to build some kind of product around PyUseCase and TextTest. I really believe in the approach Geoff has built for testing rich client python GUIs, and I'd like to see if it could be adapted for testing web applications. I have some ideas I'd like to try out, but I'll need to find real customers with real applications and problems to try them out on. I'm hoping that will be possible through the agile testing consulting that I'll be doing.

I'd also like to develop the idea of the coding dojo as a forum for teaching Test Driven Development and related agile engineering practices. I'm certain there is more that could be done to help people to get going with these skills. I'm looking into what courses are currently offered by local training providers, and hoping to both teach and develop those courses. I'm also working on my own formal training course based around the JDojo@Gbg meetings I led last year.

As you may have noticed from previous posts, I really enjoy going to conferences, and often speak at them. I don't expect that to change much :-) Geoff and I are both scheduled to speak at Agile Testing Days in Berlin in October, which I am really looking forward to. It'll be a chance for us to learn from some of the best in our industry, and share some of our ideas. Geoff's testing tools are developing all the time, and I'll be talking about what we've learnt from the many dojo meetings going on in Göteborg. I'll be speaking together with my friend and former colleague Fredrik Wendt, a stalwart member of GothPy and assistant leader of JDojo@Gbg.

So it's an exciting time for me, and I have several activities lined up to get me going with my new business. I'm also hoping to find a bit more time to spend writing articles for this blog. We'll see if I succeed!

* eLabs is a very young company with only about 8 employees, and after I agreed to join CJ and his team back in January the company strategy changed a little. After I started in March, my role didn't work out the way I'd anticipated. CJ and I had a good talk about it, and I think it's with no hard feelings on either side that I left the company at the end of June.

Programmer Conferences are like Games

Why are there so few women programmers? That's a big question. How about a related one that's slightly smaller: Why do so few women go to programmer conferences?

Nordic Ruby on Twitter
Nordic Ruby was clearly appreciated by many of its attendees. See tweets like this:

ronge: #nordicruby - best conference ever, looking forward meeting you all next year ! Lots of food for thoughts. Really sad it's over.

skanev: #nordicruby was just awesome. Thank you guys

walming: Got so much inspiration. Big thanks @elabs for #nordicruby conference.

Very few Women
What I also noticed was, that of around 100 delegates, only 2 were women*.

I have to say, I go to a lot of conferences, which gives me plenty to compare it to. In addition to Nordic Ruby, in the past year I have been to: Scottish Ruby Conference, Scandinavian Developer Conference, JFokus, Smidig, Europython and XP2009. In general, I really enjoy conferences, and none of those I've mentioned had a huge proportion of females. Nordic Ruby was not exceptional in that respect. However, although I enjoyed Nordic Ruby, it does not feature in my all-time favourite list. I'll come to why in a minute. A lot of things about the conference were very good, of course. Some of the talks were excellent, and the venues, food and parties were absolutely top knotch.

The format of of the conference was 30 minute talks (all on one track) interspersed with 30-120 minute breaks. The last session of each day was open and any attendee could give a short "lightning" talk, and many people did so. Every speaker, lightning or otherwise, had a large audience, since there was nothing else on the programme.

Hampton Catlin's talk - the two kinds of Games
My favourite talk was one by Hampton Catlin, talking about how to make applications attractive to their users. He talked a bit about the different kinds of games that people prefer. Perhaps I can expand this idea to explain why I don't rate Nordic Ruby as highly as some of the other attendees clearly did.

Hampton explained that computer games lie on a scale from Male-Oriented to Female-Oriented. They are called by those names because your physical gender is a good predictor of which sort you will prefer. (He stressed that you should keep in mind that people are complex, defy easy categorization, and a given individual could have preferences anywhere on the scale.)

The Male-Oriented game will let you score points and rank yourself against opponents. The Female-Oriented game will let you build supportive social networks with collaborators, and become admired by your peers. Hampton said that most computer games are Male-Oriented. He highlighted some exceptions, including Farmville, which is a popular game on Facebook. In fact, he said Facebook itself can be seen as a Female-Oriented game.

Programmer Conferences are like Games
This got me thinking about the Nordic Ruby conference. If Facebook can be seen as a game, can you see a conference that way too? Do attendees play for "score" and "rank"? Is the programmer's conference game so Male-Oriented that most women just aren't interested in playing?

The Conference as a Male-Oriented Game
If a programmer conference is a Male-Oriented Game, it will provide you with opportunities to improve your rank and score compared to other attendees. For example, giving a talk will let you show off the cool software project(s) you have created/contributed to. You can improve your rank by criticising other people's code, and contrasting it with the beauty of your own. You can also score "geek points" by making gratuitous references to obscure programming languages, advanced mathematics and classic sci-fi films.

Your overall conference success is measured by how many people subsequently download your open source projects, and how many followers you gain on Twitter.

The Conference as a Female-Oriented Game
If a programmer conference is a Female-Oriented Game, it will provide you with opportunities to form supportive social networks and gain admiration. Lecture-style talks aren't so good for that, so the conference will schedule sessions for attendees to have conversations with each other, and collaborate. The conference programme will raise discussion topics that interest attendees, and encourage idea sharing. There may be organized group sessions where you share programming-related problems, pool your ideas and collectively come up with strategies to move forwards. You will gain admiration by being insightful, charming and subtly drawing people's attention to your open source projects, while also being admiring of others' projects.

Your overall conference success is measured by how many people subsequently contribute to your open source projects, and how many friendly messages you get on Twitter.

Who Won Nordic Ruby?
Ok, I'm stretching the analogy rather, (!) but I'd say the Nordic Ruby conference game was a little too Male-Oriented for my liking. The focus of the programme was on lecture-style talks, and, put it this way, the speakers made way too many references to Star Wars! There were long breaks, which gave many opportunities for conversation, but there were no formal network-building activities. There was lots of time for chatting, but no mechanism to draw people together around, say, a discussion topic, or a collaborative coding exercise.

The conferences I have enjoyed most have involved relatively few lecture-style talks, and largely comprised of workshops, coding dojos, tutorials, conversation corners and open space discussions. Next week I'm going to XP2010 (which will be my seventh XP conference :-D), and it's the first ever GothPyCon this Saturday. At both I am organizing coding dojo sessions - collaborative excercises in collective learning and mutual appreciation. Bring on the Female-Oriented conference games!

* There was also two other females there, but neither are programmers.

Python and Ruby and small methods

I've been working full time in Ruby now for about a month, and I think I'm beginning to get the hang of it. In many ways it's not so different from Python.

I took some of the code I talked about struggling with in my last post, and set about translating it into Python. I wanted to see if the bugs were more obvious in a language I was more familiar with. Unfortunately before I was finished with the translation, my machine died and refused to restart, complaining about a disk error... so I may well have lost all my work. (ARRRGH!!) Anyway, before that happened, I was getting the feeling that the bugs weren't very obvious, even in a language I knew better.

What I did find though, was that it was quite hard to make the methods as short in Python as they were in Ruby. I could make them as short, the language has the necessary features to do it, but when I did so, they just stopped looking like good Python to me.

The other thing that I found was that RSpec lets you write really readable and well organized tests. Translating them into unittest made them just a travesty of their former selves.

I've just listened to this talk by Gary Bernhardt, who is experienced in both Python and Ruby, and who has clearly thought quite hard about these kinds of issues and knows both languages very well. He has even tried to write a RSpec clone for Python, called Mote. (He says himself that it isn't as good as RSpec, and that it is because Python lacks blocks, and won't let him monkeypatch core classes).

Anyway, about half way through the talk, Gary shows this code example, first in Python:

'\n'.join(obj.name
    for obj in (
        repository.retrieve(id)
        for id in ids)
   if obj)

Then the equivalent code in Ruby:

ids.map do |id|
 repository.retrieve(id)
end.compact.map do |obj|
 obj.name
end.join('\n')

Gary makes the point that the Ruby code is easier to read - you can follow it from top to bottom and see what it does. The thing is, I don't think many people would write code like that in Python. I might write it more like this:

objs = [repository.retrieve(id) for id in ids]
objs = filter(lambda x: x, objs)
names = [obj.name for obj in objs]
'\n'.join(names)

This code is four statements long, rather than one, and has two local variables ("objs" and "names") which the other two code snippets lack. In real code, you would probably be able to come up with rather more descriptive names, drawn from the problem domain. When I compare this code with the Ruby, I don't think it is any less readable. The filter(lambda x: x, objs) is not as nice as the Ruby call to "compact", but on the other hand, I think the two additional local variables make it clearer what is going on.

I'm wondering whether the trouble I was having locating bugs in these small methods was because they were cramming so much into one statement, and almost completely lacking in local variables. That seems to be the Ruby way of doing things - maybe I will just get used to it and learn to read it just as well eventually? I guess I am going to find out!

Anyway, I'm really hoping the friendly support technicians manage to save the contents of my hard disk, I want to use the code in an exercise at the upcoming Gothenburg Python Conference - GothPyCon. I am hoping to run a workshop where half the room gets the buggy code with small methods, and the other half gets the same code refactored into longer methods. You get half an hour to find the bugs, then swap to the other codebase and find them again. Then I was hoping to have a discussion about which codebase was easier to debug, and/or split into pairs and re-implement the code from scratch, and see if we could come up with other designs that solved the problem more elegantly and transparently.

If issues like this interest you - design, refactoring and testing -, I really hope you will come along!

Small methods, TDD and defects

Last week at Scottish Ruby Conference I chatted with Brian Marick about software design. The week before that, he had been in Göteborg for Scandinavian Developer Conference, and had spent a morning pair programming with Geoff on TextTest. I took the chance to ask Brian what he thought about text-based testing now that he had seen it in action.

Brian's view seems to be that text-based testing may be effective as a testing technique, but it just doesn't offer the design benefits you get with standard TDD. It doesn't give you guidance about small-scale design decisions, or intice you to structure your code into really small methods and classes. He thought the TextTest codebase wasn't bad, but that the methods were larger than he would prefer, some classes were doing too much, and some ideas were not expressed clearly.

I've previously read about this design ethic of having really really small classes and methods in Bob Martin's book "Clean Code". Bob recommends one or two line methods. In contrast, I believe Steve McConnell's "Code Complete" advocates methods small enough to fit comfortably on one screen. Geoff's design for TextTest seems to land at about 5 or 6 lines for a typical method, which is somewhere in between.

Brian said the main drawback of code structured largely into small classes and one and two line methods is that it is harder for people who are unfamiliar with the codebase to get to grips with it. You get lost in the trees, and can't easily get an overview of the forest. The big benefit is that people who are familiar with the code can potentially make sweeping improvements through very small, localized changes.

I've had the opportunity to work on this kind of codebase recently, and my experience hasn't been entirely positive. Just as Brian predicted, I've found it hard to get into the code and grasp what it is doing. All the methods are one or two lines, and call each other. My pair and I ended up creating a temporary file where we pasted a whole call chain of about 6 methods so we could see them all at once, and read them in the order they called each other. There were 3 defects hidden in that 15 or so lines of code, and it took us a day or so to identify and fix them all. Yet, the code had been TDD'd, and the methods were well named. On the surface it looked very good. It actually took me some time to convince myself that we genuinely had found a defect, and that we hadn't just misunderstood what the code was supposed to do.

The trouble seemed to stem from the fact that almost all the tests were for the "happy path" and there were several edge cases they never considered. Also, in one case the test had stubbed out the answer that one of the lower-level methods would return, and provided an answer the real code never gave. It took a long time for us to add missing tests for edge cases, and localize the defects to particular methods in the long call chain.

I'm very interested in whether we would have found it easier to find the defects if the code had been structured as two or three 5 line methods instead of 6 one and two line methods. I'm also interested if we would have found the issues more easily if the code had been built with text-based testing, with fewer, more coarse grained tests, and a few log statements printing key intermediate values. I'm considering getting the original versions of the files out of git and refactoring them to see how else they could have looked.

I don't want you to conclude that I am against building designs with really small methods, or TDD, or using stubs or anything like that. I think there is value in all these techniques, each can be done well or badly, and you make tradeoffs when you choose your approach. You still need design skills and testing skills, whether you're doing TDD or text-based testing. If Brian Marick had built TextTest the design might well have turned out differently. I don't know how much of that would have been because of his use of TDD, and how much because of his skill, and views on design.

I'm actually relishing the prospect of working on more code with very small methods, and using TDD to build on it. I've got loads to learn about software design and testing :-)

Discussion on TDD at SDC2010

As I mentioned in my last post I chaired a fishbowl discussion at SDC2010 with title "Should a professional developer always use Test Driven Development?". I was delighted that the invited panelists Michael Feathers, Geoff Bache and Andrew Dalke all turned up, along with a few dozen other conference participants. As I predicted, we had a lively and interesting debate.

Michael half-jokingly complained that Bob Martin goes around making these controvertial statements all the time, which Michael then gets to go around defending. Michael has a much more conciliatory attitude than Bob, and his take was that every truly professional developer must have at least given TDD a good try and learnt the technique, even if they then decide not to use it.

Geoff's main point was that we need to widen the definition of TDD to include any process that involves checking in tests at the same time as the code, and not restrict it to just the classic Red-Green-Refactor style with tests in the same language as the code.

Michael was largely receptive to this view, or at least that the soundbite description of "never write any code until you have a failing test" probably was a bit too brief description to encompass the whole of TDD. He did argue though, that the classic TDD style leads to code with good design characteristics of high cohesion, loose coupling, small classes and methods etc, and that he had not found other design techniques which led to better code than TDD. He was not keen to move to a TDD approach without unit tests, and lose these benefits, even if they result in good tests.

Andrew argued that TDD is not sufficient by itself to produce a good suite of tests, and that there are other, better ways to produce these tests. Andrew pointed out that he had examined Fitnesse, a codebase that Bob Martin, (and some others), has created using TDD, and that he found several bugs, including security holes in it. Michael's counterargument was that with TDD, you get as good tests as you are capable of - if you are not skilled/aware of security issues, then you won't test for security holes, whatever process you use to create tests.

Another argument of Andrew's was that he often likes to write tests that he expects to pass, to verify that his code works as expected, for example that he has implemented an algorithm correctly. In the narrow definition of TDD, you are only allowed to write tests you expect to fail. Michael's take was that this was indeed a too narrow definition of TDD. He said that he frequently writes tests as a way of asking questions of his code, and this often leads to tests that pass straight away.

Some of the "audience" also stepped up to the microphones and joined in. Brian Marick pointed out that forcing yourself to write the test first was a very good way of ensuring you do actually write the test, instead of being lazy and just writing more code. The counter to that was along the lines of that there are other processes for arriving at a good test suite, which took different kinds of discipline. Andrew quoted the sqlite project, which boasts 100% branch coverage of their code by their test suite. Publishing your coverage figures and refusing to let them slip is a way of preventing developer laziness too.

Brian Marick wrote an article about coverage and tests over a decade ago, so he summarized it for us, which was interesting, but I think slightly beside the point. I think he was trying to argue that measuring coverage alone is not enough to guarantee you have a good test suite, but I don't think that was what Andrew was trying to claim. Simply doing TDD is not a guarantee that you will end up with a good test suite either.

For me, the interesting outcome of the discussion was pointing out that the alternatives to TDD are not only "cowboy coding" or "test later, ie never", or "bad tests", but that there are other legitimate ways to come up with a good test suite, and professional developers may choose to use them instead of classic TDD. TDD is a discipline which all professional developers should perhaps have in their repertoire though. I think we agreed it is also a teaching aid for learning to write good tests.

Happily, we definitely all agree that creating a good automated test suite alongside code is important. The precise method a professional developer should always use to produce it was not agreed upon though.

Should a professional developer always use TDD?

I'm really looking forward to Scandinavian Developer Conference, and in particular the fishbowl discussion I'll be moderating on the Tuesday at 10:30am. Presenting their views will be Michael Feathers, Andrew Dalke, and Geoff Bache, and the topic under discussion is the same as the title of this post: Should a professional developer always use TDD?

I've been enthusiastic about writing automated tests for my code since 2000 when I discovered eXtreme Programming, and started using JUnit. It's become a habit for me to write tests before code. Occasionly I decide not to, perhaps I am feeling lazy, or think a test would be too difficult to write. I find I usually regret it and end up writing a test afterwards anyway.

One of the things Bob Martin, (a colleague of Michael Feathers), says about TDD in his book "clean code", is that it is a matter of professionalism. Developers should be like doctors. Would you trust a doctor who didn't wash her hands because she didn't belive in it? Well, you shouldn't trust a developer who doesn't use TDD because she doesn't believe in it.

I've known Andrew Dalke since 2002, and we've worked together on and off since then. Recently he wrote this article criticising TDD. Andrew does not believe TDD is necessary for good development work to happen. Is he unprofessional? Far from it.

My experience of working with Andrew tells me that he is an excellent programmer, who produces high quality code and automated tests. However, the process by which he arrives at this code and tests is not TDD. Tests get written during development, but not in advance of the code they test. The tests do not in any way drive the design, in fact, he uses knowledge of the design of the code to inform what tests he writes.

Andrew says in his article "Once I have a good sketch of how the code is going to be, I often continue by filling in the details. At this point unit tests starts to be useful" he likens what he does to an XP spike solution, except that he does not throw away the spike code and start over when he starts adding tests.

The other person I know who has a complex relationship with TDD is my husband Geoff. Several years ago he was labelled a heretic and almost thrown out when he admitted to a room full of XP enthusiasts that he didn't write unit tests at all. Geoff does write tests - a lot of tests in fact - but they are not xUnit tests, and they don't drive the design of his code.

Geoff uses an approach he calls "text-based testing" which involves driving the program from the command line, (or some kind of script), and having his code write a plain text log file of what it is doing. A tool called TextTest picks up the log output and compares it to the saved version from a previous run. Differences are flagged as test failure.

It's a simple idea, but it is actually very effective and easy to use when you get the hang of it. The main advantage over ordinary TDD is that there is little or no code written per test, meaning less code to maintain overall. The fact that the tests are independent of the design of the code makes refactoring easier, and writing tests for legacy code relatively risk-free.

TDD is a bit different with the text-based approach though. Geoff thinks of what he does as TDD, but actually, only half of the test is nailed down in advance of the code - only the part that tells the program which features to exercise. The part that asserts that it did the right thing is simply recorded after the code is written.

So I expect a fascinating and lively discussion to ensue when I get these guys together! Perhaps you'll join us?

(Note: I wrote up the discussion in my next post)

developer-in-test role and tester role

In my current assignment, I'm taking the role of "developer-in-test". I'm working in a large distributed development project, which is building new functionality on a large existing codebase. In practice, I work closely with the developers in the project and build automated tests for subsystems that previously had only manual tests. The developers can use these tests to support their work, and add new tests as they build new features.

My background is basically as a developer, so I have been reading up on testing. I found "Lessons Learned in Software Testing" by Kaner, Pettichord and Bach very helpful, and "Agile Testing" by Lisa Crispin and Janet Gregory helpful and also very thorough. I find it interesting that the authors of the first book started out as developers and now classify themselves as testers, while Lisa and Janet apparently always have called themselves testers, although they clearly write a fair amount of code as part of their work.

Dave Nicolette recently made a blog post "Merging the developer and tester roles" where he argues that Tester is just a specialization of Developer in the agile world, like DBA (DataBaseAdministrator) is a specialization of Developer. He argues that agile teams need to be staffed with generalizing specialists. That means anyone can turn their hand to any task that is currently needed, while still having some tasks they perform with more skill than others.

I like Dave's viewpoint, it fits my experience. I can only write effective 2nd Quadrant tests, (business facing, support the team), if I understand what the developers need, and I do that best if I have done some development on that part of the system myself. To put it another way, I need to be just as competent at writing code as the other developers in the project I'm working in, but I also need additional skills to do with testing.

I like the term "developer-in-test" to describe a role writing and enabling 2nd Qudrant tests.

Having said all that, I'm not sure I agree with Dave that the Developer and Tester roles should always be merged. In my current assignment I'm also helping a group of testers, usability experts, technical writers and product owners to get going with exploratory testing. This testing falls into Q3 of the agile testing quadrants, and is quite different. You still need testing skills, but developer skills are mostly irrelevant. It's much more about understanding what the user is trying to achieve with the system, and how they view it.

I think there is a role for non-coding testers in Q3 testing. However, I don't think you'll get far with Q3 unless you have the other quadrants well covered with automated tests. So I think the majority of work for a tester in an agile environment is still going to involve test automation. Only the biggest projects will be able to afford to have non-coding testers.