Disciplined Agile

The Disciplined Agile Delivery (DAD) process framework includes an explicit Inception phase – sometimes called a project initiation phase, startup phase, or iteration/sprint zero – which is conducted before actually starting to build a solution.  The goals of this phase include: clarifying the business problem that needs to be solved; identifying a viable technical solution; planning your approach; setting up your work environment and team; and gaining stakeholder concurrence that it makes sense to proceed with investing in the implementation of the chosen strategy.  These goals are listed in the following diagram.

In the Disciplined Agile Delivery book we devoted a lot space to describing how to effectively initiate a DAD project.  Unfortunately in our experience we have seen many organizations that are still new to agile treat this phase as an opportunity to do massive amounts of upfront documentation in the form of project plans, charters, and requirements specifications.  Some people have referred to the practice of doing too much temporary documentation up front on an agile project as Water-Scrum-Fall.  We cannot stress enough that this is NOT the intent of the Inception phase.  While we provide many alternatives for documenting your vision in Inception, from very heavy to very light, you should take a minimalist approach to this phase and strive to reach the stakeholder consensus milestone as quickly as possible.

According to the 2013 Agile Project Initiation survey the average agile team invests about 4 weeks performing project initiation activities, including initial requirements envisioning, initial architecture envisioning, forming the team, initial release planning, and so on.  Of course this is just an average, some respondents reported investing less than a week to do so and some reporting investing more than two months – the amount of time required varies depending on the complexity of the effort, your stakeholders’ understanding of their requirements, your team’s understanding of the solution architecture, whether this is a new solution or merely a new release of an existing solution, and many others.

If you are spending more than a few weeks on this phase, you may be regressing to a Water-Scrum-Fall approach.  It takes discipline to be aware of this trap and to streamline your approach as much as possible.  You can do this in several ways:

1. Recognize that the goal is to get going in the right direction.  For any project or product of reasonable complexity you want to spend a bit of time up front ensuring that you understand the problem and have what you believe to be a viable strategy to addressing that problem.  This doesn’t imply that you need a comprehensive requirements specification, a detailed project plan, nor a comprehensive architecture model but it does mean you need to do a bit of initial thinking and organizing.  In a small handful of cases, typically at scale, you may find that your team does require detailed models and plans but this is a very uncommon exception (regardless of what traditional THEORY may claim).
2. Educate people that details can safely come later.  If you have the ability to plan or model something in detail today, won’t you also have that same ability a few months from now when you actually need those details?  Of course you do.   In lean software development they recommend deferring decisions – including planning decisions,  detailed design decisions, and even requirements – until the most appropriate moment.  The observation is that by waiting you can make a better decision because you have better information at hand.  This strategy of course assumes that you’re able to overcome basic logistical problems such as having the appropriate people available at the time to help explore an issue, provide requisite information, and make the decision.   It’s far less risky, and far less expensive, in most cases to address basic logistical issues than it is to apply the process band-aid of writing detailed documentation at the beginning of a project.
3. Promote a sense of urgency.  This is the most important thing that you can do.  Just as there is risk associated with not sufficiently thinking about your strategy for approaching a new project or product there are also risks associated with doing too much up front work.  My experience is that far too many IT professionals are complacent regarding the risks associated with the project initiation activities of the Inception phase.   The longer you put off building a consumable solution the greater the risk that you’re building the wrong thing.
4. Keep your modeling efforts as light as possible.  You very likely need to do some initial requirements envisioning and architecture envisioning early in the project lifecycle to help you think through what you’re doing.  But in most cases this modeling should be high level and light, not detailed and heavy.  In every project I’ve ever been involved with the team has been asked to identify what they’re going to deliver (at least giving a rough sense of the scope) and how they’re going to do it (at least providing a rough idea of the technical strategy) to secure funding for construction.  In short, they need to do a bit of up front thinking.  You will often find that you need to reign-in some of your staff who are experienced with traditional approaches to modeling and specification.  These people have a lot of value to add to your project, modeling is an important skill needed on disciplined agile teams, but they may need help keeping their approach light-weight and incremental.   The details can come later.  See the process goals Identify Scope and Identify Technical Strategy for more thoughts on this subject.
5. Keep your planning efforts as light as possible.  Similarly you need to invest some time in high-level release planning to answer basic questions such as how long do you believe (roughly) it will take to get a release of your solution deployed and how much (roughly) this will cost.  Planning details can come throughout the Construction phase when it’s more appropriate to invest in such decisions.

I think that it’s very clear that the secret to keeping Inception short is to have the discipline to know that you need to invest some time thinking your approach through but that you want to avoid getting bogged down in too many details.  You need the discipline to do some planning but not too much.  You need the discipline to do some modeling but not too much.  You need the discipline to get going in the right direction knowing that the details will come out in time.

Whether you like it or not, as you adopt agile you will constrained by the organizational ecosystem, and you need to act accordingly.  It takes discipline to work with enterprise professionals such as enterprise architects, data administrators, portfolio managers, or IT governance people who may not be completely agile yet, and have the patience to help them.  It takes discipline to work with your operations and support staff in a DevOps manner throughout the lifecycle, particularly when they may not be motivated to do so.

Despite the fact that governing bodies such as project management offices (PMOs), architecture and database authorities, and operations may indeed be a source of impediments to your DAD adoption, these authorities serve important functions in any large enterprise.  Therefore a disciplined approach to proactively working with them and being a positive change agent to make collaboration with them more effective is required.

Although it takes discipline to work in an enterprise aware manner, the payoff can be dramatic.  Enterprise architects can help guide your team early in the lifecycle to adopt a viable architecture strategy.  Portfolio managers can help to govern your team to ensure that you truly are spending the stakeholder’s investment wisely.  Enterprise administators, including data management professionals and operations professionals, can help you to leverage existing legacy assets.  These are just a few examples of the potential ways that enterprise professionals can help a DAD team work more effectively.

I was recently asked a question around the scope of the DAD lifecycle and thought that I would share my answer publicly.

The focus of DAD is the delivery portion of the solution lifecycle, from initiation to delivery into production (or the marketplace in the case of a commercial product).  Any given solution will go through the delivery portion of the lifecycle many times during it’s existence as releases are put into production.

BUT, this isn’t the entire solution lifecycle as you in the following figure.  For example:

1. There are some portfolio management activities before a project starts such as project identification and selection that are outside the scope of DAD.  We show this as input into the DAD lifecycle to help provide context.
2. There are also post-delivery activities, particularly operations and support, that are part of the overall solution lifecycle but outside of the delivery portion of the lifecycle.  In DAD we explicitly show feedback coming back from the production portion of the solution lifecycle because this is a common occurrence.

Note that these comments apply to both the basic (Scrum-like) version of the lifecycle as well as the advanced/lean version.  Because DAD explicitly recognizes that your process improvement activities will include changes that affect the lifecycle, we don’t enforce a single DAD lifecycle.

DAD teams focus on producing repeatable results, such as delivering high-quality software which meets stakeholder needs in a timely and cost effective manner.  DAD teams do not strive to follow repeatable processes.  The observation is that because each DAD team finds themselves in a unique situation, to be most efficient they need to follow a unique process tailored to reflect that situation.  That “unique process” may be comprised of a relatively standard lifecycle and common practices such as architecture envisioning, database regression testing, non-solo development, and many others (granted, those practices may be tailored to reflect the situation too).  The point is that each team in your organization may follow a different process, albeit processes which share similar components defined by a common process framework, while achieving the results required of them.

This may of course be heresy in some organizations.  The danger with “repeatable processes” is that they grow in size over the years to address all possible situations, and as a result address none of them very well.  Imagine a project team that is large and has regulatory compliance concerns.  This team will tailor its practices accordingly.  Now imagine a small team that doesn’t have to address regulatory concerns.  An organization focused on repeatable processes might have that team follow the same process that the previous team followed, even though some of the practices had been tailored to meet scaling factors that don’t apply.  In other words, the repeatable process included some aspects that were overkill for the second team, thereby impacting their ability to deliver in a timely manner or in a cost efficient manner.  In the vast majority of organizations, when given the choice, stakeholders prefer to spend the money wisely and have the solution delivered in a timely manner, not to have the team follow a consistently “repeatable process.”

Despite some agilists reluctance to admit that projects go through phases the DAD process framework explicitly recognizes that they do.  Building serious solutions requires a lot more than just doing the cool construction stuff.  It takes discipline to ignore this rhetoric and frame your project within the scope of a full delivery lifecycle.  The Agile (Scrum-based) and Lean (Kanban-based) DAD lifecycles explicitly depict:

1. Pre-delivery activities.  There are portfolio management activities which occur long before your project begins, including the initial identification of potential projects, their prioritization, and finding initial funding for the Inception phase.
2. Three-phase delivery lifecycle.  Projects have phases that they go through.  All efforts are initiated at some point, all of them go through a construction effort (or a configuration effort in the case of purchased solutions), and hopefully some sort of deployment effort.  This is why the DAD lifecycles include explicit Inception, Construction, and Transition phases to respectively address these aspects.  We’ve confirmed via surveys that the agile teams invest time in project initiation/inception activities, often referred to as Sprint 0 or Iteration 0, and time performing release/transition activities.  From a product point they will go through at least the Construction and Transition phase many times throughout the life of the solution.
3. Post-delivery activities.  The fact that your solution is operated and supported in production, or in the marketplace for commercial products, is included.  We do this to reflect the DevOps reality many DAD teams are in the position that they are working on a new release of an existing solution, and therefore are very likely to be getting defect reports and enhancement requests coming in about previous versions.  As a result they require the discipline to treat these things as potential new requirements and act accordingly.

Without a doubt construction is an important aspect of the overall Disciplined Agile Delivery process, but it’s not the only aspect.  Yes, for many people this is the fun part of delivery, it certainly is for me.  But the reality is that as development professionals we need to explicitly consider more than just construction if we’re to be effective.  It takes discipline to adopt a broader lifecycle that goes beyond the fun stuff that we would prefer to focus on.