Disciplined Agile

Techniques that shorten the time between doing something and getting feedback about it are generally lower risk and result in lower cost to address any changes than techniques with longer feedback cycles.  Many of these techniques require agile team members to have new skills and to take a more disciplined approach to their work than they may have in less-than-agile situations.  There are several common ways to shorten the feedback cycle that are common to agile software development that are adopted by the DAD process framework.  These techniques, listed in order of immediacy, are:

1. Non-solo development.  Non-solo development strategies such as pair programming and modeling with others provide feedback within seconds.  These techniques are great strategies for reducing the feedback cycle within your team but they often require initial discipline to adopt because it can be difficult to break your former solo working habits.
2. Active stakeholder participation.  It can require significant discipline to work closely with your stakeholders, to seek and then respect their opinions, and to allow them to set important aspects of your project direction.  Working closely with stakeholders typically has a feedback cycle on the order of seconds when they are co-located with your to hours or days when you need to wait to interact with them.
3. Continuous integration.  Building, regression testing, and potentially running your work through code analysis on a continuous basis is a fairly straightforward concept which provides feedback on the order of minutes.  Doing it in practice, and more importantly the habit acting on the feedback provided from the tests and code analysis requires discipline to adopt at first because you often want to work on the next thing instead of cleaning up the work on what you’re currently doing.
4. Continuous deployment. By regularly deploying into more complex environments – to your project integration environment from your individual environment, from your project environment to your demo or independent testing environments – you put yourself in a position to receive more meaningful feedback.  Continuous deployment requires you to have the discipline to have multiple environments, to work with people external to your team (such as stakeholders and independent testers), and to seek and act on their feedback.
5. Short iterations. The length of an iteration defines the feedback cycle between promising your stakeholders you would do a bundle of work, the end result of your iteration planning session, to demonstrating what you actually got done.  It requires significant discipline to work in short iterations.  The average agile team has construction iterations of two weeks, although some teams have shorter iterations and some advanced teams have evolved beyond iterations to take a lean approach.  Then again some agile teams, particularly those at scale, may have slightly longer iterations.  The shorter the iteration the greater the discipline required to make it work because you will need to adopt many, if not all, of the techniques listed in this section.  You will also require the discipline to identify, and then address, wasteful activities that add little or no value in your current process.
6. Short release cycles.  The length of your release cycle defines the feedback cycle from promising stakeholders to deliver a new release of a solution to actual use by end users in production.  The feedback from real users is the key information to determine if you’ve delivered the right thing for them.  All other stakeholder feedback is merely an approximation up until that point.  As with short iterations, it requires increasing discipline to move from annual to bi-annual to quarterly to monthly to weekly or even daily releases into production.

This posting was modified from Chapter 21 of the forthcoming book Disciplined Agile Delivery to be published in June by IBM Press.

Yesterday Mark and I double teamed on a webcast overviewing DAD to a university class.  After the webcast we got the following question:

I am wondering if it would make sense to entirely eliminate the word “phase” from DAD’s vocabulary. What about simply talking about different types of iteration? For instance, inception could become something like “pre-construction iterations”, construction could become “construction iterations”, and transition “transitions iterations”…  That may be a bit cumbersome, but the word phase does scare people away.

Yes, the word phase might scare some people away.  We’ve thought long and hard about the terminology that we use and were also a bit worried about the word “phase” at first.  For some reason phase has become one of those dirty words in the agile community, along with other words such as governance or documentation.  The primary concern seems to be that phase implies a serial approach, something many agilists believe to be foul and evil.  Yet the reality is that projects do in fact proceed in a serial manner.  There are some project initiation activities at the start.  Then there are construction iterations/sprints one after the other in yes, a serial manner.  Then there is an effort to deploy/release your solution into production.  This is clearly “serial in the large”.

But, why the term phase?  Why not iteration?  Or season for that matter season (Gary Evans has a very coherent argument for that term)?  Because phase is accurate (albeit an agile swear word).  In the various IT surveys that I’ve run over the years I’ve found that the average agile team spends about 4.5 weeks performing initiation efforts, has construction iterations that are about 2 weeks in length, and takes an average of roughly 4 weeks to release their solution into production.  So perhaps these initiation and release efforts could each be thought of as two iterations (on average, your mileage may vary) but they really aren’t iterations in their own right usually.  Maybe they’re different length iterations?  There are several ways of thinking about it, but notice how the application of the term iteration doesn’t really make perfect sense.  Then we have numbering issues.  Is the initiation effort iteration/sprint 0?  If it’s two iterations would they be -1 and 0?  Sigh.

One thing that we have done which some may feel to be radical is we’ve chosen to adopt phase names from Unified Process (UP) – Inception, Construction, and Transition.  We could have made up different names, such as Initiation, Development, and Release respectively.  Or adopted Start Up, Construction, and Hardening (yuck, there’s more to Transition than hardening and frankly I would consider a late “hardening” effort to be a clear sign you’ve likely got a process problem you need to deal with).  And I have no doubt that there are many other options for each phase name.  Whatever names we choose someone isn’t going to be happy, and why not give a bit of a nod to a proven software development framework such as UP?

Another interesting thought is that by having named phases it makes it clearer where potential governance points in the lifecycle are.  In the diagram below you can see how several of the suggested light-weight milestones – Stakeholder vision, Sufficient functionality, and Delighted stakeholders – corresponds to the end of a phase.  Or perhaps more accurately, fulfilling the milestone is an indication that your team is moving from one phase to the next.

In the end, I think we’re pretty clear that when you tailor DAD that you can use whatever terminology you like.  In fact one financial institution that adopted an early version of the DAD basic lifecycle, the one based on Scrum, reworked the diagram to use Scrum terminology.  I think it’s a bit goofy but it works for them.

The term phase might in fact scare a few people away.  But, it’s hard to imagine that anyone that concerned about what something is named will be flexible enough to be successful at DAD anyway.

DAD teams focus on producing repeatable results, such as delivering high-quality software which meets stakeholder needs in a timely and cost effective manner.  DAD teams do not strive to follow repeatable processes.  The observation is that because each DAD team finds themselves in a unique situation, to be most efficient they need to follow a unique process tailored to reflect that situation.  That “unique process” may be comprised of a relatively standard lifecycle and common practices such as architecture envisioning, database regression testing, non-solo development, and many others (granted, those practices may be tailored to reflect the situation too).  The point is that each team in your organization may follow a different process, albeit processes which share similar components defined by a common process framework, while achieving the results required of them. 

This may of course be heresy in some organizations.  The danger with “repeatable processes” is that they grow in size over the years to address all possible situations, and as a result address none of them very well.  Imagine a project team that is large and has regulatory compliance concerns.  This team will tailor its practices accordingly.  Now imagine a small team that doesn’t have to address regulatory concerns.  An organization focused on repeatable processes might have that team follow the same process that the previous team followed, even though some of the practices had been tailored to meet scaling factors that don’t apply.  In other words, the repeatable process included some aspects that were overkill for the second team, thereby impacting their ability to deliver in a timely manner or in a cost efficient manner.  In the vast majority of organizations, when given the choice, stakeholders prefer to spend the money wisely and have the solution delivered in a timely manner, not to have the team follow a consistently “repeatable process.”

A few days ago I wrote about ranged burndown charts. Interestingly, if you track the ranges over time you end up with a chart such as the one below which corresponds to the estimating cone of uncertainty (depicted by the dashed lines).  It’s interesting to note that this example includes two common occurrences that you’ll see.  First, during iterations one and two the gross and net velocities were the same because no new functionality had been identified yet, resulting in an unranged estimate.  Second, iteration eight had a very small net velocity because the amount of new functionality was almost as much as the amount implemented, giving a huge estimation range due to the small net velocity.

Previously I discussed the difference between gross velocity and net velocity and now I’d like to show why they’re important.  A ranged burndown chart, an extension to normal burndown charts which apply both the gross and net velocity, is shown below.  Where a burndown chart uses the (gross) velocity to predict a potential end date, and by extension gives a feel for the potential project cost, a ranged burndown gives a potential range of end dates/costs.  Giving a ranged estimate is a known best practice in the IT community.

Because it’s possible that functionality can be dropped from a release part way through a project, perhaps because of a major shift in strategy or in an effort to hit a desired date, the net velocity will exceed the gross velocity that iteration.  In this case our advice is the use the change in requirements from the previous iteration to calculate the net velocity.

Note that this blog posting is excerpted from Chapter 10 of the book Disciplined Agile Delivery.