Disciplined Agile

Techniques that shorten the time between doing something and getting feedback about it are generally lower risk and result in lower cost to address any changes than techniques with longer feedback cycles.  Many of these techniques require agile team members to have new skills and to take a more disciplined approach to their work than they may have in less-than-agile situations.  There are several common ways to shorten the feedback cycle that are common to agile software development that are adopted by the DAD process framework.  These techniques, listed in order of immediacy, are:

1. Non-solo development.  Non-solo development strategies such as pair programming and modeling with others provide feedback within seconds.  These techniques are great strategies for reducing the feedback cycle within your team but they often require initial discipline to adopt because it can be difficult to break your former solo working habits.
2. Active stakeholder participation.  It can require significant discipline to work closely with your stakeholders, to seek and then respect their opinions, and to allow them to set important aspects of your project direction.  Working closely with stakeholders typically has a feedback cycle on the order of seconds when they are co-located with your to hours or days when you need to wait to interact with them.
3. Continuous integration.  Building, regression testing, and potentially running your work through code analysis on a continuous basis is a fairly straightforward concept which provides feedback on the order of minutes.  Doing it in practice, and more importantly the habit acting on the feedback provided from the tests and code analysis requires discipline to adopt at first because you often want to work on the next thing instead of cleaning up the work on what you’re currently doing.
4. Continuous deployment. By regularly deploying into more complex environments – to your project integration environment from your individual environment, from your project environment to your demo or independent testing environments – you put yourself in a position to receive more meaningful feedback.  Continuous deployment requires you to have the discipline to have multiple environments, to work with people external to your team (such as stakeholders and independent testers), and to seek and act on their feedback.
5. Short iterations. The length of an iteration defines the feedback cycle between promising your stakeholders you would do a bundle of work, the end result of your iteration planning session, to demonstrating what you actually got done.  It requires significant discipline to work in short iterations.  The average agile team has construction iterations of two weeks, although some teams have shorter iterations and some advanced teams have evolved beyond iterations to take a lean approach.  Then again some agile teams, particularly those at scale, may have slightly longer iterations.  The shorter the iteration the greater the discipline required to make it work because you will need to adopt many, if not all, of the techniques listed in this section.  You will also require the discipline to identify, and then address, wasteful activities that add little or no value in your current process.
6. Short release cycles.  The length of your release cycle defines the feedback cycle from promising stakeholders to deliver a new release of a solution to actual use by end users in production.  The feedback from real users is the key information to determine if you’ve delivered the right thing for them.  All other stakeholder feedback is merely an approximation up until that point.  As with short iterations, it requires increasing discipline to move from annual to bi-annual to quarterly to monthly to weekly or even daily releases into production.

This posting was modified from Chapter 21 of the forthcoming book Disciplined Agile Delivery to be published in June by IBM Press.

Clearly mainstream agile practices such as Scrum and Extreme Programming (XP) require discipline.  For example, effective Agile teams have the discipline to:

• Hold short, focused, and to the point daily coordination meetings rather than infrequent and time consuming status meetings.  It requires discipline to keep these meetings focused on coordination activities and thereby short and to the point.
• Commit to delivering a set of work items each iteration rather than letting deadlines slip.  It requires discipline to consistently fulfill the promises that you make to your stakeholders.
• Remove impediments in a timely fashion rather than procrastinating in pursuing a solution.  It requires discipline to tackle tough issues that are easier to ignore in the short term.
• Take the time to write tests before code rather than writing code,  It takes discipline to consistently work in a test-first manner instead of leaving testing to some time in the (distant) future.
• Test to the best of their ability instead of throwing artifacts over the wall to testers or reviewers.  It takes discipline to actively take responsibility for the quality of your own work.
• Reflect on the team’s experiences and improve their processes proactively rather than relying on process dictated by project managers or external governance bodies.  It takes discipline to stop and take time to reflect on how well your team is working and then act to improve it.
• Have a continuously working, integrated, and tested solution rather than waiting to do so when you’re “done” at the end of the lifecycle.  It takes discipline to stop all work when the build is broken so that it is repaired and the state of working, high quality solution is restored.
• Work together in a common area rather than in comfortable but isolated workspaces. It takes discipline to work effectively in a team, to do so in a respectful and trusting manner.
• Collaborate constantly with the stakeholders or their representative(s) to ensure that their expectations are met.  It takes discipline to accept that it isn’t your place to define the requirements or set priorities, particularly when you believe that you know better.
• Create and evolve deliverable documentation continuously throughout the project.  It takes discipline to accept that there’s more to successfully solution delivery than producing potentially shippable software.
• Self organize and plan the team’s work amongst themselves rather than relying on a traditional project manager to define, estimate, and assign work. It takes discipline to take responsibility for your own work and to respect the collective decisions of your team.

In our next few blogs we’ll discuss how Disciplined Agile Delivery builds on these practices to take discipline to the next level within the Enterprise.

Why does DAD not have an explicit role for an Analyst?  Without question classically trained analysts bring much needed soft skills and a structured approach to requirements elicitation and negotiation which may not be present in the other roles such as a product owner or a developer.  However, having these skills alone is not enough in an agile and lean world.

Unfortunately, professional organizations tend to encourage us to seek specialization and certifications over being cross-functional team members, which will be far more effective and valuable in the future.  This is not to say that these organizations do not deliver value in developing and maintaining standards of professional conduct and capability.  Attaining certifications (that require some degree of commitment and experience beyond a 2-day workshop) demonstrate commitment to a professional specialty.  This is admirable but I would suggest that this base of knowledge is just the start of being an effective team member on an agile project.  We should look outside our area of specialty to learn all we can about other aspects of software development.

It is my belief that in the near future, analysts will need to be competent testers if they intend to prosper in their profession.  An increasingly important skill is the ability to write requirements as executable tests.  My advice to analysts is to learn as much as you can about agile testing and seek opportunities to write your requirements as tests wherever possible.

For Business Analysts that are not interested in moving more toward the testing end of the spectrum there is another way to go.  Analysts can be good Product Owners, representing the customer on the project and by managing the scope and priorities.  In this role they can use their elicitation and facilitation skills to gain a clear understanding of what the customer needs (vs. wants).

Another potential career path is for the BA to move more into the area of traditional management consulting.  Often it is the business process that needs to be fixed, and this is where traditional Business Analysis skills will always be needed.

In my opinion, one career path for analysts is going the way of the dinosaur though.  And unfortunately this career path is often the status quo.  Traditional projects expect Business Analysts to document business processes and requirements in batch up-front in a linear, waterfall fashion.  They then must obtain sign-offs before actually proceeding with implementing any of the requirements.  Subsequent changes to those requirements are discouraged, unless through a formal, time consuming and wasteful Change Request process.  This model clearly is flawed, and eventually most companies will change their approach.  High ceremony and bureaucratic organizations such as government will be the slowest to adapt, but private companies in a competitive environment will adapt their requirements capture approach to a more agile alternative or risk being left behind by their competitors that will be more “agile” in adapting both their business processes and the solutions that support them.

Yesterday Mark and I double teamed on a webcast overviewing DAD to a university class.  After the webcast we got the following question:

I am wondering if it would make sense to entirely eliminate the word “phase” from DAD’s vocabulary. What about simply talking about different types of iteration? For instance, inception could become something like “pre-construction iterations”, construction could become “construction iterations”, and transition “transitions iterations”…  That may be a bit cumbersome, but the word phase does scare people away.

Yes, the word phase might scare some people away.  We’ve thought long and hard about the terminology that we use and were also a bit worried about the word “phase” at first.  For some reason phase has become one of those dirty words in the agile community, along with other words such as governance or documentation.  The primary concern seems to be that phase implies a serial approach, something many agilists believe to be foul and evil.  Yet the reality is that projects do in fact proceed in a serial manner.  There are some project initiation activities at the start.  Then there are construction iterations/sprints one after the other in yes, a serial manner.  Then there is an effort to deploy/release your solution into production.  This is clearly “serial in the large”.

But, why the term phase?  Why not iteration?  Or season for that matter season (Gary Evans has a very coherent argument for that term)?  Because phase is accurate (albeit an agile swear word).  In the various IT surveys that I’ve run over the years I’ve found that the average agile team spends about 4.5 weeks performing initiation efforts, has construction iterations that are about 2 weeks in length, and takes an average of roughly 4 weeks to release their solution into production.  So perhaps these initiation and release efforts could each be thought of as two iterations (on average, your mileage may vary) but they really aren’t iterations in their own right usually.  Maybe they’re different length iterations?  There are several ways of thinking about it, but notice how the application of the term iteration doesn’t really make perfect sense.  Then we have numbering issues.  Is the initiation effort iteration/sprint 0?  If it’s two iterations would they be -1 and 0?  Sigh.

One thing that we have done which some may feel to be radical is we’ve chosen to adopt phase names from Unified Process (UP) – Inception, Construction, and Transition.  We could have made up different names, such as Initiation, Development, and Release respectively.  Or adopted Start Up, Construction, and Hardening (yuck, there’s more to Transition than hardening and frankly I would consider a late “hardening” effort to be a clear sign you’ve likely got a process problem you need to deal with).  And I have no doubt that there are many other options for each phase name.  Whatever names we choose someone isn’t going to be happy, and why not give a bit of a nod to a proven software development framework such as UP?

Another interesting thought is that by having named phases it makes it clearer where potential governance points in the lifecycle are.  In the diagram below you can see how several of the suggested light-weight milestones – Stakeholder vision, Sufficient functionality, and Delighted stakeholders – corresponds to the end of a phase.  Or perhaps more accurately, fulfilling the milestone is an indication that your team is moving from one phase to the next.

In the end, I think we’re pretty clear that when you tailor DAD that you can use whatever terminology you like.  In fact one financial institution that adopted an early version of the DAD basic lifecycle, the one based on Scrum, reworked the diagram to use Scrum terminology.  I think it’s a bit goofy but it works for them.

The term phase might in fact scare a few people away.  But, it’s hard to imagine that anyone that concerned about what something is named will be flexible enough to be successful at DAD anyway.

In writing the book on DAD, Scott and I focused on a describing a pragmatic approach to being agile.  An alternative approach could have been to describe “how to do agile” planning, modeling, requirements, etc. which could have come off as dogmatic and prescriptive.  Or we could have shown all the agile practices described elsewhere, and said “you choose”.  Other methods that have tried to provide guidance to every possible situation have in the past proven to be overwhelming and difficult to adapt.

Instead, based on what really happens in enterprises that adopt agile we chose to describe different strategies to achieve your solution goals and where different, sometimes non-agile practices might make sense.  Sometimes there are situations that require us to use strategies that are not particularly agile.  How do we adapt to be as agile as we can to achieve our goals while dealing with enterprise realities such as compliance and vendor management?

We do have opinions about which strategy is most effective in certain situations and we realize that your approach might need to be different depending on your circumstances.  While we do suggest a path to maximizing your agility capability, DAD provides a flexible framework to adopt agile practices accordingly.  In this way we feel that our book is different than the other agile books out there that may paint an idealistic picture that is not realistic for your organization or project.  As an example, here is an excerpt from the upcoming book where we describe various strategies for choosing a your release cadences:

—

Table 10-5. Comparing production release cadences.

We recommend aiming for a release cadence of no more than six months, with a goal of getting it down to three months or shorter.  We recognize that organizations new to agile may find the concept of releasing every six months difficult at first, particularly when length release processes during the Transition phase still exist.  Strive to keep your Inception phase to two weeks or less, which can be hard if key stakeholders are unavailable or your organization still has lengthy milestone review processes.  During Construction, we recommend aiming for iterations of two weeks for co-located teams and of no more than four weeks DAD teams working at scale. We also recommend that construction iteration lengths remain the same, although if you’re a team new to agile that you should consider longer iterations at first and then as you gain experience with agile techniques and learn to collaborate effectively that you shorten your iterations over time.  We prefer having an internal release at the end of each iteration, although once you’ve fully adopted the practice of continuous deployment (CD) we recommend making your builds available to others whenever they’re successful.  Finally, strive to keep the Transition phase to be less than or equal to the length of a single Construction iteration, ideally shorter.

—–

We hope that you find that this approach to covering DAD gives you practical, actionable guidance for adapting agile to your organization.