coding is like cooking: Code Kata

For a little while now I've been collecting Refactoring Kata exercises in a github repo, (you're welcome to clone it and try them out). I've recently facilitated working on some of these katas at various coding dojo meetings, and participants seem to have enjoyed doing them. I usually give a short introduction about the aims of the dojo and the refactoring skills we're focusing on, then we split into pairs and work on one of these Refactoring Katas for a fixed timebox. Afterwards we compare designs and discuss what we've learnt in a short retrospective. It's satisfying to take a piece of ugly code and after only an hour or so make it into something much more readable, flexible, and significantly smaller.

Test Driven Development is a multifaceted skill, and one aspect is the ability to improve a design incrementally, applying simple refactorings one at a time until they add up to a significant design improvement. I've noticed in these dojo meetings that some pairs do better than others at refactoring in small steps. I can of course stand behind them and coach to some extent, but I was wondering if we could use a tool that would watch more consistently, and help pairs notice when they are refactoring poorly.

I spent an hour or so doing the GildedRose refactoring kata myself in Java, and while I was doing it I had two different monitoring tools running in the background, the "Codersdojo Client" from http://content.codersdojo.org/codersdojo_client/, and "Sessions Recorder" from Industrial Logic. (This second tool is commercial and licenses cost money, but I actually got given a free license so I could try it out and review it). I wanted to see if these tools could help me to improve my refactoring skills, and whether I could use them in a coding dojo setting to help a group.

Setting it up and recording your Kata
The Codersdojo Client is a ruby gem that you download and install. When you want to work on a kata, you have to fiddle about a bit on the command line getting some files in the right places, (like the junit jar), then modify and run a couple of scripts. It's not difficult if you follow the instructions and know basicly how to use the command line. You have a script running all the time you are coding, and it runs the tests every time you save a source file.

The Sessions Recorder is an Eclipse plugin that you download and install in the same way as other Eclipse plugins. It puts a "record" button on your toolbar. You press "record" before you start working on the kata.

Uploading the Kata for analysis
When you've finished the kata, you need to upload your recording for analysis. With the Codersdojo Client, when you stop the script, it gives you the option of doing the upload. When that's completed it gives you a link to a webpage, where you can fill in some metadata about the Kata and who you are. Then it takes you to a page with the full final code listing and analysis.

The Sessions Recorder is similar. You press the button on the Eclipse toolbar to stop recording, and save the session in a file. Then you go to the Indutrial Logic webpage, log into your account, and go to the page where you upload your recorded Session file. You don't have to enter any metadata, since you have an account and it remembers who you are, (you did pay for this service after all!) It then takes you to a page of analysis.

Codersdojo Client Analysis
The codersdojo client creates a page that you can make public if you want - mine is available here. It gives you a graph like this:

It's showing how long you spent between saving the file, (a "move"), and whether the tests were red or green when you saved. There is also some general statistics about how long you spent in each move, and how many modifications there were on average. It points out your three longest moves, and has links to them so you can see what you were doing in the code at those points.

I think this analysis is quite helpful. I can see that I'm going no more than two or three minutes between saving the file, and usually if the tests go red I fix them quickly. Since it's a refactoring kata I spend quite a lot of moves at the start where it's all green, as I build up tests to cover the functionality. In the middle there is a red patch, and this is a clear sign to me that I could have done that part of the kata better. Looking over my code I was doing a major redesign and I should have done it in a better way that would have kept the tests running in the meantime.

Towards the end of the kata I have another flurry of red moves, as I start adding new functionality for "Conjured" items. I tried to move into a more normal TDD red-green-refactor cycle at that point, but it actually doesn't look like I succeeded very well from this graph. I think I rushed past the "green" step without running the tests, then did a big refactoring. It worked in the end but I think I could have done that better too.

Sessions Recorder Analysis
The Sessions Recorder produces a page which is personal to me, and I don't think it allows me to share it publicly on the web. On the page is a graph that looks like this:

As you can see it also shows how long I spend with passing and failing tests, in a slightly different way from the Codersdojo Client's graph. It also distinguishes compiler errors from failing tests, (pink vs red).

This graph also clearly shows the areas I need to improve - the long pink patch in the middle where I do a major redesign in too large a step, and at the red bit at the end when I'm not doing TDD all that well.

The line on the graph is a "score" where it is awarding me points when I successfully perform the moves of TDD. Further down the page it gives me a list of the "events" this score is based on:

(This is just some of the events, to show you the kinds of things this picks up on.) "New Green Test" seems to score zero points, which is a bit disappointing, but adding a failing test gets a point, and so does making it pass. "Went green, but broke other tests" gets zero points. It's clearly designed to help me successfully complete red-green-refactor cycles, not reward me for adding test coverage to existing code, then refactoring it.

There is another graph, more focused on the tests:

This graph has mouseover texts so when you hover over a red dot, it shows all the compilation errors you had at that point, and if you hover over a green dot it tells you which tests were passing. It also distinguishes "compler errors" from a "compiler rash". The difference is that a "compiler rash" is a more serious compilation problem, that affects several files.

You can clearly see from this graph that the first part of the kata I was building up the test coverage, then just leaning on these tests and refactoring for the rest. It hasn't noticed that I had two @Ignore 'd tests until the last few minutes though. (I added failing tests for Conjured Items near the start then left them until I had the design suitably refactored near the end).

I actually found this graph quite hard to use to work out what I need to improve on. There seem to be three long gaps in the middle, full of compilation errors where I wasn't running the tests. Unlike with the Codersdojo Client, there isn't a link to the actual code changes I was making at those points. I'm having trouble working out just from the compiler errors what I should have been doing differently. I think one of these gaps is the same major redesign I could see clearly in the Codersdojo Client graph as a too big step, but I'm not so sure what the other two are.

There are further statistics and analysis as well. There is a section for "code smells" but it claims not to have found any. The code I started with should qualify as smelly, surely? Not sure why it hasn't picked up on that.

Conclusions
I think both tools could help me to become better at Test Driven Development, and could be useful in a dojo setting. I can imagine pairs comparing their graphs after completing the kata, discussing how they handled the refactoring steps, and where the design ended up. Pairs could swap computers and look through someone else's statistics to get some comparison with their own.

The Codersdojo Client is free to use, and works with a large number of programming languages, and any editor. You do have to be comfortable with the command line though. The Sessions Recorder tool only supports Java and C# via Eclipse. It has more detailed analysis, but for this Refactoring Kata I don't think it was as helpful as it could have been.

The other big difference between the tools is about openness. The Sessions Recorder keeps your analysis private to you, and if you want to discuss your performance, it lets you do so with the designers of the tool via a "comment on this page" function. I havn't tried that out yet so I'm not sure how it works, that is, whether you get feedback from a real person as well as the tool.

The Codersdojo Client also lets you keep your analysis private if you want, but in addition lets you publish your Kata performance for general review, as I have done. You can share your desire for feedback on twitter, g+ or facebook. People can go in and comment on specific lines of code and make suggestions. That wouldn't be so needed during a dojo meeting, but might be useful if you were working alone.

Further comparison needed
On another occasion I tried out the Sessions Recorder on a normal TDD kata, and found the analysis much better. For example this graph of me doing the Tennis kata from scratch:

This shows a clear red-green pattern of small steps, and steadily increasing score rewarding me for doing TDD correctly. Unfortunately I didn't do a Codersdojo Client session at the same time as this one, for comparison. A further blog post is clearly needed for this case... :-)

Tuesday, 20 December 2011

Global Day of Code Retreat

A little while back I was back in Stockholm facilitating another Code Retreat, (see previous post), this time as part of Global Day of Code Retreat, (GDCR). (Take a look at Corey Haines' site for loads of information about this global event that comprised meetings in over 90 cities worldwide, with over 2000 developers attending.)

I think the coding community could really do with more people who know how to do TDD and think about good design, so I'm generally pretty encouraged by the success of this event. I do feel a bit disappointed about some aspects of the day though, and this post is my attempt to outline what I think could be improved.

I spent most of the Global Day of Code Retreat walking around looking over people's shoulders and giving them feedback on the state of their code and tests. I noticed that very few pairs got anywhere close to solving the problem before they deleted the code at the end of each 45 minute session. Corey says that this is very much by design. You know you don't have time to solve the problem, so you can de-stress: no-one will demand delivery of anything. You can concentrate on just writing good tests and code.

In between coding sessions, I spent quite a lot of effort reminding people about simple design, SOLID principles, and how to do TDD (in terms of states and moves). Unfortunately I found people rarely wrote enough code for any of these ideas to really be applicable, and TDD wan't always helping people.

I saw a lot of solutions that started with the "Cell" class, that had an x and y coordinate, and a boolean to say whether it was alive or not. Then people tended to add a "void tick(int liveNeighbourCount)" method, and start to implement code that would follow the four rules of Conway's Game of Life to work out if the cell should flip the state of its boolean or not. At some point they would create a "World" class that would hold all the cells, and "tick()" them. Or some variant of that. Then, (generally when the 45 minutes were running out), people started trying to find a data structure to hold the cells in, and some way to find out which were neighbours with each other. Not many questioned whether they actually needed a Cell class in the first place.

Everyone at the code retreat deleted their code afterwards, of course, but you can see an example of a variant on this kind of design by Ryan Bigg here (a work in progress, he has a screencast too).

Of course, as facilitator I spent a fair amount of time trying to ask the kind of questions that would push people to reevaluate their approach. I had partial success, I guess. Overall though, I came away feeling a bit disillusioned, and wanting to improve my facilitation so that people would learn more about using TDD to actually solve problems.

At the final retrospective of the day, everyone seemed to be very positive about the event, and most people said they learnt more about pair programming, TDD, and the language and tools they were working with. We all had fun. (If you read Swedish, Peter Lind wrote up the retrospective in Valtech's blog) This is great, but could we tweak the format to encourage even more learning?

I think to solve Conway's Game of Life adequately, you need to find a good datastructure to represent the cells, and an efficient algorithm to "tick" to the next generation. Just having well named classes and methods, although a good idea, probably won't be enough.

For me, TDD is a good method for designing code when you already mostly know what you're doing. I don't think it's a good method for discovering algorithms. I'm reminded of the debate a few years back when Peter Norvig posted an excellent Sudoku solver (here) and people thought it was much better than Ron Jeffries' TDD solution to the same problem (here). Peter was later interviewed about whether this proved that TDD was not useful. Dr Norvig said he didn't think that it proved much about TDD at all, since

"you can test all you want and if you don’t know how to approach the problem, you’re not going to get a solution"
(from Peter Siebel's book "Coders At Work", an extract of which is available in his blog)

I felt that most of the coders at the code retreat were messing around without actually knowing how to solve the problem. They played with syntax and names and styles of tests without ever writing enough code to tell whether their tests were driving a design that would ultimately solve the problem.

Following the GDCR, I held a coding dojo where I experimented with the format a little. I only had time to get the participants do two 45 minute coding sessions on the Game of Life problem. At the start, as a group we discussed the Game of Life problem, much as Corey recommends for a Code Retreat introduction. However, in addition, I explained my favoured approach to a solution - the data structure and algorithm I like to use. I immediately saw that they started their TDD sessions with tests and classes that could lead somewhere. I feel that if they had continued coding for longer, they should have ended up with a decent solution. Hopefully it would also have been possible to refactor it from one datastructure to another, and to switch elements of the algorithm without breaking most of the tests.

I think this is one way to improve the code retreat. Just give people more clues at the outset as to how to tackle the problem. In real life when you sit down to do TDD you'll often already know how to solve similar problems. This would be a way for the facilitator to give everyone a leg-up if they havn't worked on any rule-based games involving a 2D infinite grid before.

Rather than just explaining a good solution up front, we could spend the first session doing a "Spike Solution". This is one of the practices of XP:

"the idea is just to drive through the entire problem in one blow, not to craft the perfect solution first time out."
From "Extreme Programming Installed" by Jeffries, Anderson, Hendrickson, p41

Basically, you hack around without writing any tests or polishing your design, until you understand the problem well enough to know how you plan to solve it. Then you throw your spike code away, and start over with TDD.

Spending the first 45 minute session spiking would enable people to learn more about the problem space in a shorter amount of time than you generally do with TDD. By dropping the requirement to write tests or good names or avoid code smells, you could hopefully hack together more alternatives, and maybe find out for yourself what would make a good data structure for easily enumerating neighbouring cells.

So the next time I run a code retreat, I think I'll start with a "spiking" session and encourage people to just optimize for learning about the problem, not beautiful code. Then after that maybe I'll sketch out my favourite algorithm, in case they didn't come up with anything by themselves that they want to pursue. Then in the second session we could start with TDD in earnest.

I always say that the code you end up with from doing a Kata is not half as interesting as the route you took to get there. To this end I've published a screencast of myself doing the Game of Life Kata in Python. (The code I end up with is also published, here). I'm hoping that people might want to prepare for a Code Retreat by watching it. It could be an approach they try to emulate, improve on or criticise. On the other hand, showing my best solution like this is probably breaking the neutrality of how a code retreat facilitator is supposed to behave. I don't think Corey has ever published a solution to this kata, and there's probably a reason for that.

I have to confess, I already broke all the rules of being a facilitator, when I spent the last session of the Global Day of Code Retreat actually coding. There was someone there who really wanted to do Python, and no-one else seemed to want to pair with him. So I succombed. In 45 minutes we very nearly had a working solution, mostly because I had hacked around and practiced how to do it several times before. It felt good. I think more people should experience the satisfaction of actually completing a useful piece of code using TDD at a Code Retreat.

Friday, 14 October 2011

The Clean Coder & Apprenticeship Patterns

I'm speaking next week at ScanDev on Tour in Stockholm on the subject of "Software Development Craftsmanship", and as part of my research I read both "The Clean Coder" by Robert C. Martin and "Apprenticeship Patterns" by Dave Hoover & Adewale Oshineye. These are very different books, but both aimed at less experienced software developers who want to learn about what it means to be a professional in the field. In this article I'd like to review them side by side. First some text from each preface on what the authors think the books are about:

Apprenticeship Patterns

"This book should help you through the tough decisions you face as a newcomer to the field of professional software development. " (preface xi)

The Clean Coder

"This book is about software professionalism. It contains a lot of pragmatic advice" (preface xxii)

The Content
Both books contain a lot of personal stories and anecdotes from the authors' careers, and begin with a short autobiography. Some of the advice is also similar. Both advise you to practice with "Kata" exercises, to read widely and to find suitable mentors. I think that's mostly where the similarities end though.

Dave and Ade don't say much about how to handle unreasonable managers imposing impossible deadlines. Bob Martin devotes a several chapters to this kind of issue, handling pressure, time management, estimation, making committments etc.

Dave and Ade talk more about how to get yourself into situations optimized for learning and progress in your career. They advise you to "Be the worst", "Find mentors", seek "Kindred Spirits". In other words, join a team where you're the least skilled but you'll be taught, look for mentors in many places, and get involved in the community.

Bob talks about a lot of specific practices and has detailed advice. He mentions "... pairing is the most efficient way to solve a problem" (p164) Later in the chapter he suggests the optimal composition of job roles in a gelled team. (p169) He also has some advice about how to successfully argue with your boss and go over their head when necessary (p35).

The Advice
Those few example perhaps illustrate that these two books are miles apart when it comes to writing style, approach and world view. Dave&Ade have clearly spent a lot of time talking with other professionals about their material, acting on feedback and testing their ideas for validity in real situations. The book is highly collaborative and while full of advice, is not prescriptive.

Bob Martin on the other hand loves to be specific, provocative and extreme in his advice. "QA should find nothing."(p114) "You should plan on working 60 hours per week." (p16) "Avoid the Zone." (p62) "The jury is in! ... TDD works" (p79) These are some of his more suprising pieces of advice, which I think are actually fairly doubtful propositions when taken to extremes like this. Mixed in are more reasonable statements. "You do not have to attend every meeting to which you are invited" (p123) "The professional developer is calm and decisive under pressure". (p150)

The way everything is presented as black-and-white, do-or-do-not-there-is-no-try is actually pretty wearing after a while. He does it to try to make you think, as a rhetorical device, to promote healthy discussion. I think it all too easily leads the reader to throw the baby out with the bathwater. I can't accept one of his recommendations, so I throw them all out.

Some of Dave&Ade's advice is actually just as hard to put into practice. Each of their patterns is followed by a call to action. Things like re-implementing a program you've written in an imperative language in a functional language (p21). Join or start a user group (p65). Solve the same coding exercise once a week for the next four weeks (p79). None of these things is particularly easy to do, but they seem to me to be interesting and useful challenges.

Collaboration
Bob has also clearly not collaborated very widely when preparing his material. One part that particularly sticks out for me is a footnote on page 75:

"I had a wonderful conversation with @desi (Desi McAdam, founder of DevChix) about what motivates women programmers. I told her that when I got a program working, it was like slaying the great beast. She told me that for her and other women she had spoken to, the act of writing code was an act of nurturing creation." (footnote, p75)

Has he ever actually run his "programming is slaying a great beast" thing past any other male programmers? Let me qualify that - non-fantasy-role-playing male programmers? Thought not. This is in enormous contrast to Dave&Ade, whose book is full of stories from other people backing up their claims.

Stories
Bob's book is full of stories from his own career, and he is very honest and open about his failures. This is a very brave thing to do, and I have a great deal of respect for him for doing so. It's also really interesting to hear about the history of what life was like when computers filled a room and people used punch cards to program them. Dave&Ades stories are less compelling and not always as well written.

Bob's book is not just about his professional life, he shares his likes and dislikes. He reccommends cycling or walking to recharge your energy, or "focus-manna" as he calls it, (p127). Reading science fiction as a cure for writer's block. (p66) Listening to "The Wall" while coding could be bad for your design. (p63) When describing "Master" programmers he likens them to Scotty from Star Trek. (p182)

All this is very cute and gives you a more rounded picture of what software professionalism is about. Maybe. Actually it really puts me off the idea. I know a lot of software developers like science fiction and fantasy role playing, but it really isn't mandatory. He usually says that you may have other preferences, and you don't have to do like he does, but I just don't think it helps all that much. The rest of the book is highly dogmatic about what you should and shouldn't do, and it kind of rubs off.

Conclusions
The bottom line is, I wouldn't reccommend "The Clean Coder" to any young inexperienced software developer, particularly not if she were a woman. There is too much of it written from a foreign culture, in a demanding tone, propounding overly extreme behaviour. The interesting stories and good pieces of advice are drowned out.

On the other hand, I would recommend "Apprenticeship Patterns". I think it is humbly written and anchored in real experience from a range of people. I agree with them when they say you need to read it twice to understand it. The first time to get an overview, the second time to understand how the patterns connect. It's not as easy to read as it might be. But still, I think the content is interesting, and it gives a good introduction to what being a professional software craftsman is about, and how to get there.

Thursday, 25 August 2011

Refactoring Kata fun

I've been working on a kata called "Tennis"*, which I find interesting, because it is quite quick to code, yet is a big enough problem to be worth doing. It's also possible to enumerate pretty much all the allowed scores, and get very comprehensive test coverage.

What I've found is that when I'm using TDD to solve the Kata, I tend to only enumerate actually a very small number of the test cases. I generally end up with something like:

Love-All
Fifteen-All
Fifteen-Love
Thirty-Forty
Deuce
Advantage Player1
Win for Player1
Advantage Player2

I think that's enough to test drive a complete implementation, built up in stages. I thought it would be enough tests to also support refactoring the code, but I actually found it wasn't. After I'd finished my implementation and mercilessly refactored it for total readability, I went back and implemented exhaustive tests. To my horror I found three (of 33) that failed! I'd made a mistake in one of my refactorings, and none of my original tests found it. The bug only showed up with scores like Fifteen-Forty, Love-Thirty and Love-Forty, where my code instead reported a win for Player 2. (I leave it as an exercise for the reader to identify my logic error :-)

So what's the point of TDD? Is it to help you make your design good, or to protect you from introducing bugs when refactoring? Of course it should help with both, but I think doing this practice exercise showed me (again!) that it really is worth being disciplined and careful about refactorings. I also think I need to develop a better sense for which refactorings might not be well covered by the tests I have, and when I should add more.

This is something that my friend Andrew Dalke brings up when he criticises TDD. The red-green-refactor iterative, incremental rhythm can lull you into a false sense of security, and means you forget to stop and look at the big picture, and analyze if the tests you have are sufficient. You don't get reminded to add tests that should pass straight away, but might be needed if you refactor the code.

So in any case, I figured I needed to practice my refactoring skills. I've created comprehensive tests and three different "defactored" solutions to this kata, in Java and Python. You can get the starting code here. You can use this to practice refactoring with a full safety net, or if you feeling brave, without. Try commenting out a good percentage of the tests, and do some major refactoring. When you bring all the tests back, will they still all pass?

I'm planning to try this exercise with my local python user group, GothPy, in a few weeks time. I think it's going to be fun!

* Tennis Kata: write a program that if you tell it how many points each player has won in a single game of tennis, it will tell you the score.

Friday, 3 December 2010

Django

Last night at GothPy we had a play with Django, a web application framework for Python. I'm fairly familiar with Rails, so it was interesting to see a different take on solving the same kinds of problems.

I downloaded Django for the first time when preparing for the meeting, and spent about a day going through the tutorial and trying stuff out. At the meeting were a few people who have used Django before, notably Mikko Hellsing, who has worked with it daily for several years.

It was so much easier learning the tool from people who knew it than by reading the documentation. Constant code review and real time answering of questions is just brilliant for learning.

At the meeting we decided to implement a very very simple web application, similar to the one that Johannes Brodwall performed as a Kata at XP2010 together with Ivar Nilsen. He calls it "Java EE Spike Kata" since he does it in Java with no particular web frameworks, just what comes with Java. (There is a video of him doing it on his blog, and sample solution here on github).

I thought we should be able to implement the same little application in any web framework, and it might be interesting to see differences, so I though we should try doing it in Django. It just involves two pages. One page "Add User" which lets you create a new user and save it to the database, and another page "Search User" which lets you search for users, and presents results. So the scenario is to create a user, search for them, and see they are returned.

When I work on a problem in Rails I usually start with a Cucumber scenario for the feature, and I discovered there is a python version of Cucumber called Lettuce. We could have course have just used Cucumber with python, but given the big "WARNING - Experimental" notice Aslak wrote on this page, I thought we could give Lettuce a try.

So at the meeting we all worked together with one laptop and a projector, (me at the keyboard), and we started with a Lettuce scenario. We implemented step definitions using Django's test Client, which is a kind of headless browser that understands how to instrument a Django application. Then we spent a couple of hours writing Django code together until the scenario was all green.

The code we ended up with isn't much to write home about, but I've put it up on github here.

What we learned from this exercise
Of course since I know Rails much better, I found it interesting to compare the two web frameworks. Django seems similar in many ways. It was dead easy to get up and running, it gives you a basic structure for your code, and separates the model from the presentation layer.

The O-R mapping looks quite similar to ActiveRecord in the way you declare model classes backed by tables in the db. The presentation layer seems to have a different philosophy from Rails though. The html view part is rather loosely coupled to your application, and doesn't allow you to embed real python code in it, just basic control structures.

You hook up the html to the controller code using url regular expression matching. I was a little confused about exactly how was supposed to work, since what I considered controller code was put in a file called "views.py". Most of the code we wrote ended up in here, and I feel a bit unhappy with it. It seems to be a mixture of stuff at all levels of abstraction. The Django Form objects we used seemed quite powerful though, and reduced the amount of code we had to write.

The biggest difference I noticed compared with Rails was how explicit Python is about where stuff comes from. You always have to declaratively import or pass stuff before you can use it, so I found it straightforward to follow the connections and work out which code was executed and where it came from. I think that is because of Python's philosophy of strict namespaces, which Ruby doesn't have so much of.

I also liked the way Django encourages you to structure a larger web application into a lot of independent smaller ones, and lets you include and re-use other people's small apps. Rails has plugins too of course, but I thought Django's way made it seem very natural to contribute and re-use code.

Comparing Lettuce to Cucumber, they look almost the same, (by design). One small difference I found was that Cucumber step definitions don't care if you write "Given", "When" or "Then" in front of them, whereas Lettuce did. So I had steps like this:


Then I should see "results found"
And I should see "Name"

where the first step passed and the second step was reported as unimplemented by Lettuce. So Lettuce need a little more work to be really usable.

I was also pretty disappointed by the Django test client for implementing test steps. It seemed to interact with pages at an abstraction layer lower than I am used to - at the level of making post and get requests and parsing html as a dom. I missed Capybara and its DSL for interacting with web pages. I couldn't find any equivalent. I probably should have turned to Selenium directly, but since we had no client side javascript it seemed overkill. (Francisco Souza has written about using Lettuce with Selenium compared with the Django test client here).

When it comes to unit-level testing, Django provides an extension to the normal unittest tool that comes with python (unittest was originally based on JUnit). We didn't do much with it at the session, and it seemed to work fine. It's nothing like RSpec though, and I miss that expressiveness and structure for my tests whenever I work in Python.

Overall
All in all it was fun to look at Django with a group and to get some really helpful comments and insights from people who know it far better than I do. The Kata we chose seemed to work ok, but I should really do it again in Rails since I spent the whole time comparing Django with Rails in my head, not Django with Johannes' Java code :-)

My conclusions are basically that Django looks like a good alternative to Rails. It would take time to learn, and surely has strengths and weaknesses I can't really evaluate from one short session looking at it. However, I'd fairly sure I'd have to do some work improving the testing tools if I was going to be happy working with it for real.

Monday, 1 November 2010

KataMontyHall

I was at the local Ruby User Group last night, and I coded up KataMontyHall (see below) as a prepared Kata in front of the group. I got some great comments and feedback while I was coding, and I think the solution I ended up with was better than any I had created by myself during my practice sessions. I also got some new ideas about different approaches which I plan to try out and see if they improve the code even more.

Dave Hoover was visiting from the US, on his way to a speaking engagement at Öredev. He and I were both surprised to discover a common interest in the Monty Hall Dilemma. Dave has previously worked on it, and he showed us some code that he wrote 5 or 6 years ago when he was still learning Ruby. He even wrote an online version of the game when he was first learning ajax, that you can play yourself!

The Monty Hall Dilemma
There is a gameshow hosted by Monty Hall where contestants try to win a big prize, which is behind one of three doors. The contestant begins by choosing a door, but not opening it. Then Monty steps forward and opens one of the other doors. He reveals a goat (!). Then the contestant has the choice of either sticking with the door they have already chosen, or switching to the other unopened door. Whichever door the contestant decides on will be opened, and if they find the prize, they get to keep it. (I'm not sure what happens if they get the second goat!) So what's the best strategy? Stick or switch?

People are biased towards sticking with what they've chosen, and the vast majority of people stick with the door they choose originally. Intuitively there should be an equal chance of the prize being behind any of the three doors, so it shouldn't matter if you stick or switch. However, in this case, your intuition is wrong. You are twice as likely to win the prize if you switch to the other unopened door.

I am not the only one to think this result is incorrect, apparently famous mathematicians have also refused to accept it. What finally convinced them, was a computer simulation. Hmm I thought, that sounds like an interesting piece of code :-)

Pigeons are smarter than Humans
I heard about the Monty Hall Dilemma listening to the quirks and quarks podcast. Apparently humans are strange, because they don't learn to switch doors. Some researchers set up the same problem for pigeons, with birdseed behind one of the doors, and found the birds quickly learnt to switch doors. You can read the research for yourself!

So if you, like me, are keen to prove yourself more intelligent than a pigeon, why don't you spend some time writing a little program that simulates the game? If you get it right, your version of KataMontyHall should clearly show that switching doors is the best strategy :-)

Friday, 24 September 2010

Coding Dojo Survey Results

A little while ago, Fredrik Wendt and I put together a little survey and sent it to all the people we know who have attended coding dojo meetings here in Göteborg. There are several groups in the city which run these kinds of meetings from time to time. We wanted to know what kinds of things people are learning from practicing code katas in a group, and get some feedback and comments to help us to develop the groups we run.

The survey got 29 responses - thankyou to everyone who participated! I was very encouraged to see that the overwhelming majority - 90% - said they would recommend the dojo to other people who wanted to improve their programming skills. There were lots of comments too - I won't include them all - but I particularly liked:

"Dojos make me relive the joy of red-green-refactor”

“you learn a lot from each other at these meetings”

“... planting seeds that have improved me as a programmer and a problem solver”

“I have started to think in terms of TDD”

The first question asked people which dojo(s) they had been to.

Fredrik and I set up JDojo@Gbg about a year ago, as an free coding dojo for Java programmers, sponsored by Iptor. We advertized it as a taught course - 5 sessions, once a month, come and learn TDD. We ran it twice together, and now Fredrik is continuing it together with Ola Berg.

The second sort of coding dojo going on in the town is various language user groups. It must be four years since I pestered Niclas Nilsson and CJ that we should do code Katas at Got.rb (Göteborg Ruby User Group). Inspired by the success of that group, and together with Andrew Dalke and Johan Lindberg, I set up GothPy (Gothenburg Python User Group) about two years ago. More recently, Jonas Nicklas set up Got.js (Göteborg javascript user group) and Jörgen Lundberg set up Scala Geats (Göteborg Scalaenthusiaster). So there are several small, language specific groups that meet and do dojo evenings occasionally, interspersed with other activities.

The third kind of coding dojo meetings going on is where people run them internally at their employer, which lots of people do from time to time. In addition, Fredrik and I are both facilitating these kinds of groups at clients, as part of our work coaching teams and encouraging them to be more agile.

I was pleased that people from all these sorts of groups filled in the survey, and that many people had been to more than one kind.

Most of the survey respondents had been to 4 or 5 dojo meetings:

Roughly how many dojo meetings have you been to?

We asked several questions about perceived skill level before and after participation in dojo meetings. We wanted to know what people thought they were learning, basically.

As you can see, people were fairly good at the programming language in question before the dojo(s), and only reported marginal improvements, 0.3 on average, on a five point scale.

When it comes to the IDE used in the dojo, again, we are looking at a modest improvement, also 0.3 on a five point scale.

Using Mock Objects is a topic that doesn't always come up in dojo meetings, I personally have certainly had more of a focus on classic TDD. The feedback shows that people didn't feel they were that good at it before the meetings, and not that much better afterwards - an average 0.5 increase on a 5 point scale. From some of the comments, I think this is an area to focus on more in future. To that end, we're having a GothPy meeting next week looking at Mockist TDD :-)

Refactoring is an essential part of Test Driven Development, and in fact any modern programmer's toolbox. I was pleased to see that people thought the dojo had helped them to improve this skill too - 0.5 on a five point scale.

So it looks like going to a dojo encourages you to write unit tests more often - 0.6 (on a five point scale) more often on average.

This result is the most pleasing to me, since I think TDD is such an important skill, and it can be so difficult to get going with. People are moving from a low level of skill up to a medium level of skill - a whole 1 point (on a five point scale) on average! No-one is claiming to be a TDD expert yet though, I will have to repeat the survey in a few years and see if anyone is brave enough to claim that by then ;-)

Fredrik and I are off to Agile Testing Days in a week or so, where we'll be presenting a bit about what we've been doing at these various groups, and what we recommend for others starting their own coding dojos. We've learnt a lot by running these meetings, and had fun too. Hopefully these survey results might encourage you to find a coding dojo and improve your programming skills!

Friday, 30 April 2010

Python and Ruby and small methods

I've been working full time in Ruby now for about a month, and I think I'm beginning to get the hang of it. In many ways it's not so different from Python.

I took some of the code I talked about struggling with in my last post, and set about translating it into Python. I wanted to see if the bugs were more obvious in a language I was more familiar with. Unfortunately before I was finished with the translation, my machine died and refused to restart, complaining about a disk error... so I may well have lost all my work. (ARRRGH!!) Anyway, before that happened, I was getting the feeling that the bugs weren't very obvious, even in a language I knew better.

What I did find though, was that it was quite hard to make the methods as short in Python as they were in Ruby. I could make them as short, the language has the necessary features to do it, but when I did so, they just stopped looking like good Python to me.

The other thing that I found was that RSpec lets you write really readable and well organized tests. Translating them into unittest made them just a travesty of their former selves.

I've just listened to this talk by Gary Bernhardt, who is experienced in both Python and Ruby, and who has clearly thought quite hard about these kinds of issues and knows both languages very well. He has even tried to write a RSpec clone for Python, called Mote. (He says himself that it isn't as good as RSpec, and that it is because Python lacks blocks, and won't let him monkeypatch core classes).

Anyway, about half way through the talk, Gary shows this code example, first in Python:


'\n'.join(obj.name
    for obj in (
        repository.retrieve(id)
        for id in ids)
   if obj)

Then the equivalent code in Ruby:


ids.map do |id|
 repository.retrieve(id)
end.compact.map do |obj|
 obj.name
end.join('\n')

Gary makes the point that the Ruby code is easier to read - you can follow it from top to bottom and see what it does. The thing is, I don't think many people would write code like that in Python. I might write it more like this:


objs = [repository.retrieve(id) for id in ids]
objs = filter(lambda x: x, objs)
names = [obj.name for obj in objs]
'\n'.join(names)

This code is four statements long, rather than one, and has two local variables ("objs" and "names") which the other two code snippets lack. In real code, you would probably be able to come up with rather more descriptive names, drawn from the problem domain. When I compare this code with the Ruby, I don't think it is any less readable. The filter(lambda x: x, objs) is not as nice as the Ruby call to "compact", but on the other hand, I think the two additional local variables make it clearer what is going on.

I'm wondering whether the trouble I was having locating bugs in these small methods was because they were cramming so much into one statement, and almost completely lacking in local variables. That seems to be the Ruby way of doing things - maybe I will just get used to it and learn to read it just as well eventually? I guess I am going to find out!

Anyway, I'm really hoping the friendly support technicians manage to save the contents of my hard disk, I want to use the code in an exercise at the upcoming Gothenburg Python Conference - GothPyCon. I am hoping to run a workshop where half the room gets the buggy code with small methods, and the other half gets the same code refactored into longer methods. You get half an hour to find the bugs, then swap to the other codebase and find them again. Then I was hoping to have a discussion about which codebase was easier to debug, and/or split into pairs and re-implement the code from scratch, and see if we could come up with other designs that solved the problem more elegantly and transparently.

If issues like this interest you - design, refactoring and testing -, I really hope you will come along!

Sunday, 14 February 2010

XP2010 workshop and lightning talk

I've just heard that two of my proposals for XP2010 have been accepted, which means I will definitely be off to Trondheim in early June. I've heard Trondheim is very beautiful, and the XP conference it usually excellent, so I'm really looking forward to it. It will actually be my 8th XP conference!

I'm going to be running a half day workshop "Test Driven Development: Performing Art", which will be similar to the one I ran at XP2009, (which I blogged about here). I've put up a call for proposals on the codingdojo wiki, so do write to me if you're interested in taking part.

The other thing I'll be doing is a lightning talk "Making GUI testing productive and agile". This will basically be a brief introduction to PyUseCase with a little demo. Hopefully it will raise interest in this kind of approach.

Perhaps I'll see you there?

Monday, 3 August 2009

a new dojo - JDojo@Gbg

I'm planning to start a new dojo this autumn, called JDojo@Gbg. I was inspired by the guys at Responsive in Linköping, who I met at XP2009. They have been running a dojo for some time now, and find it is an excellent way to introduce programmers from their clients to the ideas of Test Driven Development. I think we could do with more test infected programmers about the place in Göteborg, too.

I already run a dojo as part of GothPy, and Got.rb also runs regular Kata/dojo evenings, but because those programming languages are not mainstream, many developers wouldn't consider coming along. That is why the new dojo is explicitly going to use Java, or at least, the JVM platform.

I'm thinking about what Katas we are going to tackle at the new dojo, and last night I had a go at KataFizzBuzz in Java. It is an extremely simple problem to solve, and initially I thought it was too easy to be a Kata actually. Then at agile2008 I was looking around for a Kata that Michael Feathers and I could perform in 4 minutes for the "Programming with the Stars" competition, and it seemed to fit the bill. I was quite pleased we got done in that short amount of time (in python of course :-)

A couple of people have commented that this Kata is actually quite good for teaching TDD, just because it is so simple to solve. People are forced to think about TDD instead of the problem. It can easily be made more interesting by adding new requirements too. So I think I might try it out at JDojo@Gbg.

Saturday, 4 July 2009

europython

I just wrote a report about europython on my company blog.

Monday, 11 May 2009

Europython

I'm looking forward to Europython in Birmingham at the end of June. Geoff and I are going to be rather busy at it. They've just published the programme, and between us we are holding 5 sessions. I'm running a "coder's dojo", a "clean code challenge", and Geoff and I are doing a tutorial on texttest together. Geoff is also giving talks about texttest and pyUseCase.

The coder's dojo session is a copy of the original XP2005 workshop by Laurent Bossavit and Emmanual Gaillot, only in python with different Katas. The structure is the same though - introduction, prepared Kata, randori, retrospective. I thought it worked really well in 2005, so why change a winning format?

The clean code challenge is an idea I came up with. I've written on this blog before about KataArgs, and my dodgy python translation of Bob Martin's code. I'm interested to know what the python community will make of it. I'm basically planning to throw this code out to anyone who turns up, and ask them to refactor it into better python. I'm of course hoping they will produce some innovative, beautifully pythonic solutions, and show me what clean code looks like in python.

The tutorial on texttest is essentially similar to the one Geoff and I did at Europython in 2005. It's longer though, a half day, and builds on all the experience we have had since, doing tutorials at XP2006 and agile2008 for example.

All in all, I hope we'll have some time and energy left to go to the other items in the programme. It looks like being a busy conference.

Wednesday, 6 May 2009

TDD performer artists announced

As I wrote in a previous post, I am organising a workshop at XP2009 called "Test Driven Development: Performing Art". I am very pleased to announce that I have four pairs of expert programmers willing to perform prepared Katas at it! You can read about it here. I am especially pleased that all the performers are experienced coders with previous involvement in coding dojos in places such as Stockholm, Linköping, London, São Paulo and Helsinki. I think we're going to have a great afternoon on 27th May.

Wednesday, 25 February 2009

Java-ish python

I am really interested to find out more about this concept of "clean code", and in particular how it relates to programming language. To this end, I'm still chewing on KataArgs.

My latest idea is to start from Bob Martin's Java implementation, and translate this as directly as possible into python. My idea is to then refactor it to be more pythonic, and see if it turns out looking anything like his Ruby implementation.

I have put up some code on launchpad, which is my attempt at a direct translation of the Java. It was really interesting to do, actually. Of course I had read the Java before, and followed along in the book all the steps to create it, but actually translating it made me understand the code on another level. When I tackled this Kata from scratch, I also got a much better understanding of it, but this was different.

One thing that jumped out at me was the error handling. It's much more comprehensive than anything I've produced in my own solutions, and also compared with his Ruby solution. So I think it's a bit misleading of him to say "I recently rewrote this in Ruby and it was 1/7 the size". Of course it is smaller. It does less. Although to be fair, in a way it does more too...

One thing I found awkward to translate was the use of the enum for the list of error codes. Python doesn't have a direct equivalent, being dynamic as it is. The other awkwardness was the Java iterator. In python iterators are built into the language, but don't let you backtrack, or get the current index, unlike Java ones. I was surprised to find how extensively the tests rely on this functionality. To my mind, they probe the internals of the args parser too much.

By far my most interesting observation, though, is the one I want to explore more. This code is well written Java, but directly translated, it makes pretty poor python. Why is that? What, specifically, are the smells, and what are the antidote refactorings?

I will no doubt post more on what I find, (with the help of my friends at GothPy, of course)

fun with KataArgs

I'm having fun with this KataArgs. In my last post, I took a closer look at Bob Martin's Java and Ruby solutions to it. Since then, we have tackled this Kata at a couple of GothPy meetings. (My coding dojo; the code is here.)

Several of us did some more work on the kata individually after the meetings, and a lively discussion on our mailing list ensued. I also challenged the local Ruby user group Got.rb to have a go at it, and one person posted his solution there too.

It's all good fun, anyway, and hopefully we're all learning something about what we mean by "clean code" along the way.