The second doctrine of Agile is having “working software over comprehensive documentation” which drives much of the obsession with being “done” in the Agile community. In other words, the whole reason for doing Agile is so that you have something workable at the end of each iteration. This makes perfect sense as the justification for breaking large chunks of work into smaller pieces is so that you can get a better handle on it, build subsequent pieces upon previously built items and change rapidly if needed. This should foster better creativity and to respond the changes and feedback quicker.
Unfortunately, this doesn’t seem to happen much in the real world as even one of the co-founders of the movement, Jeff Sutherland discusses in the Q&A on the launch of his new book “Scrum: The Art of Doing Twice the Work in Half the Time” that was just recently published, that this is often not the case:
Recently I visited 12 of the largest and most successful companies in Silicon Valley on a book tour. They average number of Scrum teams in these companies was more than 200, many much larger than that. I was impressed at the scale at which Scrum is being implemented...
Unfortunately, over 80% of these teams do not have tested, working software at the end of a sprint. This creates huge delays and many problems. Scrum becomes slow, hard, and painful. It is a gross violation of the second value in the Agile Manifesto ["Working software over comprehensive documentation"]. Therefore, it is “Bad Agile.” The Product Owners and customers cannot count on anything except being late.
I want to communicate to all teams, that for Scrum to be fast, easy, and fun as at the Ashram college, you must have working product at the end of the sprint.
My feeling is that it is because of the obsession with being “done” that causes this continuous dysfunction since it compounds the problem in an opposite direction. Let me clarify what I mean: As was just mentioned, the ideal situation is to have a working deliverable at the end of each iteration (or “Sprint” in Scrum lingo) which would drive the improvement of subsequent iterations into a form of continuous improvement we all assumingly acknowledge. This would create that whole idea of teams that gel and self-organize and at some point the customer would get what they want and stop.
But let’s look at this from an opposite direction: If we take Jeff Sutherland’s metric that 80% do not achieve the condition of being “done” at an iteration's end, it stands to reason that this problem would have the opposite effect and each failed iteration would cause more failed iterations into a detrimental cycle of continuous dysfunction. Though the ideal situation is to allow failures, my feeling is that the majority of executive management bought into the idea of Agile to avoid failures in the first place so that no failures are tolerated at all, even though failures can sometimes lead to breakthroughs. This situation would be made worse if you adopted Agile in an environment that is really not meant for it: e.g., if it was introduced into a large compartmentalized organization (note that it was metioned in the Sutherland quote that Scrum teams number 200 and above!).
So these Agile teams that fail to produce a working deliverable work even harder and as anyone knows, they would inevitably hit a point of diminishing returns and end up like that famous quote on insanity “where you do the same thing over and over again expecting a different result!” And if Silicon Valley startups are failing 80% of the time, the non-startups which mean regular companies not doing cutting edge software and technology, probably have a fail rate in the upper 90th percentile. Yikes, than that means there’s some serious dysfunction out there!
So what’s the solution? Maybe be less rigid about being “done”, but than that kind of defeats the whole agility thing. Perhaps a bit more planning to make sure the iterations produce something done? But do this too much and you end up with waterfall done in prolonged chunks. Maybe waterfall is better since you fail big and start over again. It’s the proverbial choice of whether you want to die instantly but painfully, or by being poked a thousand times which is not as painful but agonizingly prolonged.
What do you think? What's been your solution?