Disciplined Agile

We received a new book review from one of our younger readers.  His comments:

• too much text, not enough pictures
• lots of content on DAD, but not enough on MOM

Thanks for the feedback Brayden, keep it coming!

The Disciplined Agile Delivery (DAD) process framework includes an explicit Inception phase – sometimes called a project initiation phase, startup phase, or iteration/sprint zero – which is conducted before actually starting to build a solution.  The goals of this phase include: clarifying the business problem that needs to be solved; identifying a viable technical solution; planning your approach; setting up your work environment and team; and gaining stakeholder concurrence that it makes sense to proceed with investing in the implementation of the chosen strategy.  These goals are listed in the following diagram.

In the Disciplined Agile Delivery book we devoted a lot space to describing how to effectively initiate a DAD project.  Unfortunately in our experience we have seen many organizations that are still new to agile treat this phase as an opportunity to do massive amounts of upfront documentation in the form of project plans, charters, and requirements specifications.  Some people have referred to the practice of doing too much temporary documentation up front on an agile project as Water-Scrum-Fall.  We cannot stress enough that this is NOT the intent of the Inception phase.  While we provide many alternatives for documenting your vision in Inception, from very heavy to very light, you should take a minimalist approach to this phase and strive to reach the stakeholder consensus milestone as quickly as possible.

According to the 2013 Agile Project Initiation survey the average agile team invests about 4 weeks performing project initiation activities, including initial requirements envisioning, initial architecture envisioning, forming the team, initial release planning, and so on.  Of course this is just an average, some respondents reported investing less than a week to do so and some reporting investing more than two months – the amount of time required varies depending on the complexity of the effort, your stakeholders’ understanding of their requirements, your team’s understanding of the solution architecture, whether this is a new solution or merely a new release of an existing solution, and many others.

If you are spending more than a few weeks on this phase, you may be regressing to a Water-Scrum-Fall approach.  It takes discipline to be aware of this trap and to streamline your approach as much as possible.  You can do this in several ways:

1. Recognize that the goal is to get going in the right direction.  For any project or product of reasonable complexity you want to spend a bit of time up front ensuring that you understand the problem and have what you believe to be a viable strategy to addressing that problem.  This doesn’t imply that you need a comprehensive requirements specification, a detailed project plan, nor a comprehensive architecture model but it does mean you need to do a bit of initial thinking and organizing.  In a small handful of cases, typically at scale, you may find that your team does require detailed models and plans but this is a very uncommon exception (regardless of what traditional THEORY may claim).
2. Educate people that details can safely come later.  If you have the ability to plan or model something in detail today, won’t you also have that same ability a few months from now when you actually need those details?  Of course you do.   In lean software development they recommend deferring decisions – including planning decisions,  detailed design decisions, and even requirements – until the most appropriate moment.  The observation is that by waiting you can make a better decision because you have better information at hand.  This strategy of course assumes that you’re able to overcome basic logistical problems such as having the appropriate people available at the time to help explore an issue, provide requisite information, and make the decision.   It’s far less risky, and far less expensive, in most cases to address basic logistical issues than it is to apply the process band-aid of writing detailed documentation at the beginning of a project.
3. Promote a sense of urgency.  This is the most important thing that you can do.  Just as there is risk associated with not sufficiently thinking about your strategy for approaching a new project or product there are also risks associated with doing too much up front work.  My experience is that far too many IT professionals are complacent regarding the risks associated with the project initiation activities of the Inception phase.   The longer you put off building a consumable solution the greater the risk that you’re building the wrong thing.
4. Keep your modeling efforts as light as possible.  You very likely need to do some initial requirements envisioning and architecture envisioning early in the project lifecycle to help you think through what you’re doing.  But in most cases this modeling should be high level and light, not detailed and heavy.  In every project I’ve ever been involved with the team has been asked to identify what they’re going to deliver (at least giving a rough sense of the scope) and how they’re going to do it (at least providing a rough idea of the technical strategy) to secure funding for construction.  In short, they need to do a bit of up front thinking.  You will often find that you need to reign-in some of your staff who are experienced with traditional approaches to modeling and specification.  These people have a lot of value to add to your project, modeling is an important skill needed on disciplined agile teams, but they may need help keeping their approach light-weight and incremental.   The details can come later.  See the process goals Identify Scope and Identify Technical Strategy for more thoughts on this subject.
5. Keep your planning efforts as light as possible.  Similarly you need to invest some time in high-level release planning to answer basic questions such as how long do you believe (roughly) it will take to get a release of your solution deployed and how much (roughly) this will cost.  Planning details can come throughout the Construction phase when it’s more appropriate to invest in such decisions.

I think that it’s very clear that the secret to keeping Inception short is to have the discipline to know that you need to invest some time thinking your approach through but that you want to avoid getting bogged down in too many details.  You need the discipline to do some planning but not too much.  You need the discipline to do some modeling but not too much.  You need the discipline to get going in the right direction knowing that the details will come out in time.

Last week I was discussing DAD with a new client and he asked me “Is DAD just an Agile version of RUP?”  In a word, no.  DAD is a toolkit composed of a hybrid of methods and practices as shown in the diagram.  It includes the best of Scrum, Extreme Programming (XP), Agile data and modeling, and yes, the Unified Process (UP).  DAD also includes additional content such as agile governance that is not present in any of these methods.  As the diagram indicates, probably the method that adds most to DAD is XP, not the UP.
The Rational Unified Process (RUP) started as a small manageable process framework targeted specifically for building software within the context of an iterative lifecycle.  However over time, Rational (and subsequently IBM) added additional guidance and artifacts to extend the applicability of RUP to all sorts of projects, such as package implementation, maintenance projects, technology specific guidance (J2EE, .Net etc.), systems engineering and may other project types.  It became unwieldy and hard to understand and apply successfully.  In fact it is frequently misused (with the Elaboration phase often being treated as a big requirements upfront (BRUF) phase as an example).  This misuse has been described by Julian Holmes as RINO (RUP in name only).  To be clear, RUP properly applied in the right circumstances can be very effective.  Unfortunately though, that often does not happen.  One of the issues with applying the RUP framework to different types of projects is that it is described as a “Use case-driven” approach.  Specifying requirements as use cases, and then creating component-based architecture from these use case realizations is fundamental to RUP.  This presents challenges for maintenance projects or package implementations where it may not make sense to produce use cases at all.

DAD does not prescribe a use case-driven approach, or insist that OOAD be rigorously applied to build out services/components.  A use case-driven approach is a potential practice to apply but there is a danger that this could lead to an exhaustive requirements specification which is not particularly agile.  We would prefer to use a user story-driven approach if that makes sense within the context of your project.  User stories might not be the right choice either.  Perhaps you are in a regulatory environment that demands a traditional software requirements specification (SRS).  The key point is that you will have to adapt to the situation that you find yourself in.  This is why we prioritize the team’s work with a work item list comprised of work items, rather than Scrum’s backlog comprised of user stories.  Using a work item list allows us the flexibility to put any type of work onto our backlog, extending the applicability of DAD to many types of projects beyond those for which RUP or Scrum would be ideally suited.

DAD is goal-driven, not artifact-driven.  It does not prescribe practices or specific artifacts.  Rather, it suggests alternative strategies than can be applied at certain parts of the lifecycle with the pros and cons for each, but which ones you choose is up to you.

In my next post I will describe which aspects of the Unified Process did make it into DAD and why.

Whether you like it or not, as you adopt agile you will constrained by the organizational ecosystem, and you need to act accordingly.  It takes discipline to work with enterprise professionals such as enterprise architects, data administrators, portfolio managers, or IT governance people who may not be completely agile yet, and have the patience to help them.  It takes discipline to work with your operations and support staff in a DevOps manner throughout the lifecycle, particularly when they may not be motivated to do so.

Despite the fact that governing bodies such as project management offices (PMOs), architecture and database authorities, and operations may indeed be a source of impediments to your DAD adoption, these authorities serve important functions in any large enterprise.  Therefore a disciplined approach to proactively working with them and being a positive change agent to make collaboration with them more effective is required.

Although it takes discipline to work in an enterprise aware manner, the payoff can be dramatic.  Enterprise architects can help guide your team early in the lifecycle to adopt a viable architecture strategy.  Portfolio managers can help to govern your team to ensure that you truly are spending the stakeholder’s investment wisely.  Enterprise administators, including data management professionals and operations professionals, can help you to leverage existing legacy assets.  These are just a few examples of the potential ways that enterprise professionals can help a DAD team work more effectively.

The recording of my 1-hour InformationWeek webcast on DAD is available here:

https://event.on24.com/eventRegistration/EventLobbyServlet?target=registration.jsp&eventid=482700&sessionid=1&key=44991426F6D07634F67A0321147EEA90&sourcepage=register