Wednesday, 6 October 2010

Agile Testing Days - first conference day

Yesterday, Geoff gave his talk about agile GUI testing. For anyone who missed it, here is a video of him giving roughly the same talk earlier this year at Europython. Gojko Adzic has also blogged about what he said, which is exactly the kind of feedback we came to this conference for. In his post Gojko explains Geoff’s testing approach, and seems quietly positive about it, at least compared to the “sine of death” you get with other UI test automation tools. His conclusion that it looked more suitable for legacy code than greenfield development is a little uncomfortable though. We think it works there too :-)


Today I’m giving my talk about teaching Test Driven Development via Coding Dojos. I’m looking forward to some feedback from the community about that. In the meantime I’ve written a bit about the three keynote talks we listened to yesterday.


Lisa Crispin

The day started with a keynote from Lisa Crispin titled “Agile Defect Management”. The overall message was to “lower the bar!” and aim to reduce defect count as far as possible, even to the point where a defect tracking software becomes superfluous. There was a lot of talk about whether such a tool was needed, and in what situations, and she gave a good overview of the state of the art of thinking in this area.


This was a good talk, with audience involvement, by someone who knows what they are talking about. The thing is I have higher expectations from a keynote. I expect to come away inspired and challenged, with some new insight to take back to my daily life. For me, this talk didn’t really deliver that. Lisa concentrated too much on the specific question of whether to use an electronic defect tracking tool or not, and didn’t sufficiently put that question in to the wider context given in the title of the talk “agile defect management”. I was disappointed to find nothing really original or surprising in what Lisa said.


Linda Rising

This was a good talk to hold straight after lunch when everyone is a bit sleepy. Linda spoke very amusingly on the subject “Deception and Estimation: How we Fool Ourselves”. She began by inviting us to see this as “the weird talk” of the conference, and that she was going to go through some of the latest scientific research in the area of cognition and psychology and hoped to relate that to why we have such trouble making good estimates in the context of a software project.


A self-confessed scientific amateur, she stated up front that she wouldn’t provide references to the research she mentioned, although she could give them to you if you emailed her and asked. Linda then proceeded to relate a series of amusing anecdotes designed to illustrate how irrational and over-optimistic people can be. (Markus Gärtner has blogged about what they all were). Towards the end of the talk, Linda began to relate all this to the subject of estimation, and told a story about some conference where she met people who were trying to apply scientific methods, statistics and data mining to the problem of improving estimates in software projects. In my mind, a seemingly a rational response to the problem of irrational, over-optimistic people.


Linda then did what I saw as a complete about-turn in her argument. She quoted one proponent of this “scientific” approach as saying, “well we can’t just make up a number, can we?”. Well no, we can’t, Linda just spent the last half hour convincing us humans are over-optimistic and irrational, and you can’t trust them to make up numbers. Yet that seemed to be exactly what Linda then proposed we do in agile. The points about the way agile overcomes this natural human over-optimism by for example breaking down problems into thin slices, using the wisdom of the crowd and enforcing tight feedback cycles, all kind of got crammed in at the end with little or no explaination.


Quite apart from those specific criticisms of her argument, as someone with a scientific training I didn’t like the way Linda protrayed science and scientists. They initially appeared in Linda’s talk as white-coated oracles who make pronouncements of the truth. “80% of drivers think they are above average”. “You eat more at an all-you-eat buffet”. “Online daters lie about their age and weight”. She then attempted to shatter this illusion of scientific infalliability by quoting Planck, who pointed out that the scientific process doesn’t always proceed in an orderly manner, and sometimes new and better theories only really catch on when the older generation who invented the previous ones atually die off.


Yes, scientists are human too, and you do them a disservice when you only present their results as received truths, without references, and without explaining either the methods they used to reach the conclusions, or what they themselves think of the wider applicability of the results.


This could have been a far more interesting talk about actual recent research studies - what’s coming out of the latest brain imaging techniques, for exaple. Linda could also have spent more time explaining how agile works with human nature to provide better estimates and plans. For all that it made me laugh, all this talk left me with was a bunch of amusing anecdotes and an uneasy feeling that agile was anti-science.


Elisabeth Hendrickson

The last session of the day saw Elisabeth Hendrickson presenting “Lessons Learned from 100+ Simulated Agile Transitions”. With a huge amount of energy and panache, Elisabeth strode around the stage, explaining what happens in an exercise which she usually does with a group of 8-20 people over the course of a whole day. Within the framework of this simulation, she drew out stories and anecdotes to illustrate such diverse subjects as the Satir change model, how physical layout affects communication, the difference between status meetings and communication meetings, and tests as alignment tools.


This talk was definitely the highlight of my day. Elisabeth took some things I kind of knew about, and made me think about them in a different light, from a different angle, and in a new context. I was challenged to go back to my standup meetings and make sure they really are about communication, and that my task board really does make status visible. I have a additional way of explaining ATDD to people - in terms of aligning developers and other stakeholders and getting them moving in the same direction.


Having said all that, I do have a criticism (are you surprised?!). Elisabeth released her slides under the creative commons license, but she does not release the details of her simulation. This would effectively prevent anyone else from running it. I think this is rather like a tool vendor who presents a new testing approach, which by the way, you can’t use without either buying their tool, or spending a large amount of time and money developing your own version. Those kinds of talks don’t tend to get accepted at this kind of conference.


I was disappointed that Elisabeth didn’t release her simulation materials, and I’m not sure why she doesn’t want to. She is obviously a fantastic agile coach and facilitator, and has more invitations to speak than she has time or inclination to accept. It would surely only enhance her reputation to make the agile transition simulation game materials available.


Update: I talked to Elisabeth afterwards about her materials, and she related a story about another independent consultant she knows, who arrived at a client site ready to do a simulation exercise that he had designed, only to discover the participants had done the exact same exercise the week before with a different consultant! The other consultant had just taken the material without permission or acknowledgement. Elisabeth doesn't want to end up in the same situation, and I can understand that. She did say she could release more information about the simulation though, enough that you could understand how it is designed, and perhaps build your own similar one. I think that would be a reasonable compromise. I just felt slightly cheated after her keynote - I wanted to look at this simulation, poke at it, see how it works and understand why she could use it to generate so many great insights into agile transitions.


6 comments:

Lisa said...

Emily, I agree w/ your observations of my keynote talk. I've only done a few keynotes in my career, and it's definitely not my strength, I'm an extremely nervous speaker! I like interacting with a smaller group, and I really enjoyed the interactive part of my talk. I do think it's an interesting topic that people should keep thinking about. I don't think I emphasized my point enough that we should be concerned with preventing, rather than tracking, defects. I'll try to do better next time!

I'm a "worker bee" who works full time with an awesome team, and I like to share what we've learned, and what I've learned from other teams. I am so glad to see more people "from the trenches" such as yourself sharing ideas at conferences like this. I encourage everyone out there reading this blog to start your own blog, start writing, work up the courage to come present your experiences at conferences. We all benefit!

Emily Bache said...

Thanks Lisa! I'm glad you've seen my feedback and I'm sure you'll just get better and better at giving talks :-)

Like you, I like to share what I've learned and the best part of conferences is all the discussion and feedback and people telling you what they think of your ideas. Bring on the open space!

Michael Bolton http://www.developsense.com said...

Hi, Emily...

I'd like to suggest an alternative point of view to your interpretation of Elisabeth's talk.

I had a different perspective, for two reasons.

First, like Elisabeth and many other students of Jerry Weinberg, I teach experientially. The learnings from experiential exercises are often seriously weakened by specifics and knowledge of what's coming next.

Second, I have experienced the WordCount simulation myself. Elisabeth claims not to put traps into the exercise. That's true. Yet the structure of the exercise—combined with capacity for people to plant cow pies in the path ahead of them —makes staged traps unnecessary. Rather than theorizing, I can tell you from experience: Advance knowledge of the details would weaken it. One should not, in my view, lead such an exercise until one has experienced it at least once one's self. Moreover, it takes training and coaching in how to deliver the exercise well. James Bach and I have a similar policy with our exercises: you can probably lead them if you've experienced them yourself, at least once. But before that, a written description, no matter how detailed, is pretty empty and prone to misinterpretation. (I have an example that I'd be happy to lead you through online.)

So I have a apparently have criticism of your criticism (are you surprised?). :)

Cheers,

---Michael B.

jennitta said...

I am really surprised by the characterization of Elisabeth Hendrickson's keynote as a vendor talk. In my opinion, nothing could be further from the truth about this talk, or about Elisabeth herself.

During the keynote, Elisabeth shared her observations about recurring team self-organization patterns, insights about paths team take in their transition to agile, and valuable nuggets of wisdom about how to facilitate self-organizing teams during a transition.

These key messages were made more tangible and memorable because she put them into context with brief glimpses into the basic flow of her simulation. Her core message would have remained unchanged if she had not provided the contextual backdrop. At no time did I personally detect a hint of a sales pitch. She never said that these lessons could only have come from this particular simulation. Real agile transitions or any other similar simulation would have been an equally good place for a skilled eye to make these types of observations.

I wonder if the ‘vendor talk’ comment comes more from a general expectation/entitlement than from the content of the talk itself. Our industry has a generous culture of sharing (ideas, insights, software, presentations, etc), that frankly puzzles most outsiders. We should be quick with our gratitude when things are shared for the greater good, and slow to criticize when a conscious business decision is made about it.

Emily Bache said...

Michael - you may be right - you probably need to experience a simulation before you can lead it. Just like you have to experience agile before you can coach people in it. With some experience of both actual agile transitions and leading other simulations in general, some information about how Elisabeth puts this exercise together would be very interesting. Not enough to lead the same exercise perhaps - and who could possibly match Elisabeth for charisma, panache and facilitation skills anyway - but it might expand your horizons of what you attempt to teach experientially.

Emily Bache said...

Jennitta - you're quite right, the talk was full of great insights, and the simulation/game was really just a backdrop for these. At the time when I was listening, I appreciated the insights, but I guess I hadn't really thought through Elisabeths reasons for not releasing her materials. I just felt excluded. As you say, there wasn't an explicit sales pitch, it just came across as a really insight-filled, fun workshop that I had no access to, short of crossing the atlantic and paying to attend.

Since talking to Elisabeth I have more understanding of and respect for her decision to retain control of her simulation materials. At first I didn't see how it could harm her business to release them.