An important part of agile culture is to be honest and forthcoming about your mistakes, so I'd like to share one that I've made in a key diagram that exists in both our Disciplined Agile Lean Scrum Master (DALSM) courseware and in the book Choose Your WoW! This blog posting is my "failure bow" regarding mistakes that I made in the flowchart for how to choose between DA life cycles.
Figure 1 presents the original flowchart as it currently appears in the book and courseware. Don't worry, we're in the process of updating both. I'm writing this blog now because I want to make this update publicly available as quickly as possible to support people's learning journeys. There are two problems in Figure 1:
Figure 1. Choosing a DA lifecycle (original diagram).
The update to the diagram is presented in Figure 2. You can see that we've changed one of the Yes options to be No. More importantly, we've reworded the decision point so that it's clearer. We had several people point out that they didn't understand the original wording of the question about potential disruption. I had written that question from the point of view of a team composed of people with a traditional background. But, many teams now have an agile background, having gotten started with a framework like Scrum only to find it insufficient for their needs. Such teams wouldn't be disrupted, at least not very much, by adopting the Agile lifecycle. Thus we've reworked the question to instead ask about the team's agile background.
Figure 2. How to choose a DA life cycle (updated).
An important point that I would like to make about the flowchart of Figure 2 is that this is the logic that we suggest you follow, but you may still decide to make other decisions. For example, consider the decision point in the bottom-right corner. You may be working with a team that is new to agile but still decide to adopt the agile lifecycle over the lean lifecycle because you're willing to invest in the time and expense of training and coaching them in agile ways of working (WoW). Fair enough, that's your call.
I hope that this update has cleared up any confusion you may have had around this diagram.
As you know the Disciplined Agile (DA) toolkit supports several delivery lifecycles – Agile, Lean, Continuous Delivery: Agile, Continuous Delivery: Lean, Exploratory, and Program. We do this because solution delivery teams face different situations, so one lifecycle will not fit all. This begs the question of when would a team adopt each lifecycle? That is the focus of this blog.
This blog explores the following topics:
What Lifecycles Does DAD Support?
The delivery lifecycles we will explore are:
What Criteria Should You Consider When Choosing a Lifecycle?
The following table compares the lifecycles, suggesting when you would choose to follow each one.
Who Should Choose the Lifecycle?
They will often do this with the guidance of an experienced Disciplined Agile coach, particularly when they are new to DA. It’s tempting to have your portfolio management team to make this choice, and they may often make a (hopefully solid) suggestion when the first initiated an endeavour, but in the end it needs to be the choice of the team. As you see in the table above there are common considerations for when to use each lifecycle, but the primary considerations are always the skill and preferences of the team itself.
This lifecycle is a natural progression from the Agile lifecycle. Teams typically evolve to this lifecycle from the Agile lifecycle, often adopting iteration lengths of one-week or less. The key difference between this and the Agile lifecycle is that the continuous delivery lifecycle results in a release of new functionality at the end of each iteration rather than after a set of iterations. Teams require a mature set of practices around continuous integration and continuous deployment and other Disciplined DevOps strategies. This lifecycle is suitable when:
We’re often asked by people when it makes sense for a team to take a traditional approach to software development, and more generally when it makes sense on any type of project. While the focus of this article is on software development projects it also discusses non-software projects too.
Sometimes people are honestly trying to identify when each potential lifecycle, including the traditional lifecycle, makes sense to follow. In many cases this request is coming from people who are desperate to continue working in the same old way that they’re comfortable with. They often hope that the context of the situation that they’re in puts them in a position where they don’t need to adopt new ways of working. Some people get “lucky”, although how lucky it is to forgo an opportunity to gain new skills that are currently in demand is questionable at best, but most find that they need to join their colleagues in adopting agile.
Traditional development was prevalent in the 70s through 90s, but starting in the mid-90s organizations started to adopt iterative approaches such as the Unified Process (UP) as well as agile approaches such as Scrum or Extreme Programming (XP). Having said all this, the majority of organizations still have a few teams following traditional lifecycles and will often have people who believe that traditional software development is a good idea. And, as I argue in this blog, in a few situations they’re right about that.
This blog explores several issues surrounding the traditional approach:
1. What is Traditional Development?
Traditional development - also called serial, waterfall, linear, or "predictive" - is based on the idea that the delivery lifecycle should be organized into phases or stages. Figure 1 depicts a classic waterfall lifecycle, similar to what Winston Royce originally proposed (and recommended against) in his 1970 paper. In a pure waterfall the flow is unidirectional (from Requirements to Architecture to…) whereas Figure 1 depicts a “modified waterfall” using bi-directional flow that indicates feedback from a phase may go back to the previous phase.
Figure 1. The Waterfall lifecycle.
Figure 2 depicts a gated waterfall where you need to pass through a “quality gate” to move between phases. The “quality gate” tends to be based on the review and acceptance of artifacts – for example a Software Requirements Specification (SRS) coming out of Requirements, a Software Architecture Design (SAD) coming out of Architecture, and so on. It also shows how feedback can go back to any phase via a Change Control process, which on most traditional teams proves to be a change prevention process in practice. I used quotes around “quality gate” because “quality gates” tend to have very little to do with quality in practice and have a lot to do with promoting questionable bureaucracy. My point is that traditional rhetoric may be blinding you to serious deficiencies in this approach.
Figure 2. The Gated Waterfall lifecycle.
Figure 3 depicts the V-model version of the traditional approach where the test phase is depicted as multiple stages and there are clear indications of what each testing activity validates (for example, Integration Test validates your architecture).
Figure 3. The V-model lifecycle.
The governance strategy with a traditional approach tends to be artifact based (i.e. someone reviews and signs off on your architecture document) rather than risk based (i.e. we proved that the architecture works by building a vertical slice of the solution that implements high-risk requirements). Each phase tends to be staffed with specialists (i.e. requirements are captured by requirements analysts, the design is created by designers, and so on) which tends to motivate a quality gate approach, the development and maintenance of traceability information, and significant management effort to coordinate everything. This in turn increases cost, overall cycle time and decreases ability to support changing requirements (traditionalists typically fear “scope creep” whereas agilists embrace evolving requirements) which in turn decreases stakeholder satisfaction with the end product.
2. When Does Traditional/Waterfall Make Sense?
There are several situations when the traditional approach makes sense:
3. What Factors Have No Bearing on When Traditional Makes Sense?
My experience is that the following issues do not have an impact, or when they do have very little impact, on the deciding if a traditional strategy is appropriate for your team:
4. But Isn’t Traditional Better at Scale?
Not really. Agile and lean are as good or better when applied by large teams, geographically distributed teams, teams in regulatory situations, teams facing domain complexity, teams facing technical complexity, and teams facing organizational distribution. Figure 4 depicts a radar chart of potential tactical scaling factors.
Figure 4. Scaling factors faced by software development teams.
So how does traditional fair in comparison with agile and lean strategies in practice? If you poke around at the IT Surveys page you’ll see that the 2014 Software Development at Scale study found that agile teams outperformed non-agile teams across a range of success factors. Similar results can also be found in the 2006, 2008, 2010, 2011, and 2013 Project Success Surveys. The Dr. Dobb’s Journal 2008 Project Success study found that when comparing agile and traditional approaches based on level of geographic distribution that agile teams were at least as successful and often more so, on average, than traditional teams. In other words, agile was less risky. In 2006 DDJ found that agile was as good or better than traditional regardless of team size, but unfortunately I can no longer find those results online.
To be clear, some of these surveys are a bit long in the tooth. Having said that, in the past few years organizations have figured out how to successfully apply agile and lean approaches at scale. I suspect that agile and lean will both have pulled ahead even further compared with traditional at scale. We’re actively looking into these sorts of issues so stayed tuned to this blog for future research results. And, we’re not the only ones who are getting these sorts of results. Go poking around on the web and find out for yourself.
5. Does Disciplined Agile (DA) Support the Traditional Lifecycle?
The DA toolkit does not yet support a traditional lifecycle, although an upcoming release in Q2 2020 will bring some aspects of serial/traditional into play.
Having said that, DA has always recognized that in the vast majority of organizations you will have a multi-modal approach where some teams are following an agile approach, some are lean, some are taking a continuous delivery approach, and some are still following traditional. The more disciplined your organization, the more skilled your people, the less likely it is to have traditional teams in practice.
6. Doesn’t DA Support a Serial Lifecycle?
Yes, but it’s risk-based, not artifact nor activity based as you see with traditional approaches. The two project lifecycles supported by DAD, the Scrum-based Agile lifecycle and the Kanban-based Lean lifecycle, have three phases: Inception, Construction, and Transition. These phases, overviewed in Figure 5, capture the ideas that a project team needs to be initiated somehow (Inception), the solution must be developed (Construction), and then released into Production (Transition).
Figure 5. The phases and milestones of DAD project teams.
When you have stable (long-lived) product teams Inception tends to disappear (with the exception of the need for occasional modeling and planning sessions to think things through) and Transition evolves from a multi-day or multi-week phase into a multi-minute or multi-hour activity. Furthermore, the lifecycle evolves over time as you improve, moving from a phased agile/lean project lifecycle into a continuous delivery lifecycle. More on this in future blogs.
7. What About Non-Software Projects?
Although much of the advice presented above applies to non-software projects there are some differences in the non-software world. For example, in the physical construction world (think building buildings, oil refineries, highways, ...) they will often work in a traditional manner. This happens for many good reasons:
Having said that, even when it makes sense to follow a more traditional approach you can still benefit from applying agile strategies on your project. For example I was recently involved in a physical construction project and I was struck by how much of the actual work was performed in an agile manner. At a high-level it looked like a traditional/serial project, and it was, but on a day-to-day basis it was reasonably agile within the limits they had. So it was more of a hybrid strategy masquerading as traditional.
8. Why Should You Listen to Me?
Here is a brief description of my background in traditional software development:
The point is that I believe I have sufficient experience with traditional approaches to speak about where it does and doesn’t work. I hope this blog has provided some valuable insights for you.
One of the key philosophies of the Disciplined Agile (DA) toolkit is that it presents software development teams with choices and guides them through making the right choices given the situation they face. In other words, it helps them to truly own their process. Part of doing so is to choose the software delivery lifecycle (SDLC) that is the best fit for their context. In this blog posting we overview the DAD Exploratory lifecycle which is based in part on Lean Startup strategies.
This lifecycle can be applied in two ways:
The following diagram overviews the DAD Exploratory lifecycle. This lifecycle is followed by agile or lean teams that find themselves in startup or research situations where their stakeholders have an idea for a new product but they do yet understand what is actually needed by their user base. As a result they need to quickly explore what the market wants via a series of quick learning experiments.
Now let’s describe how the Exploratory lifecycle works. There are six activities to this lifecycle:
To summarize, the DAD process framework takes a flexible, non-prescriptive approach to software-based solution delivery. As a result of this philosophy DAD supports several development lifecycles, one of which is the Lean-Startup-based Exploratory lifecycle described in this posting. This lifecycle is typically followed in situations where you are unsure of what your user base wants, and sometimes even when you are unsure of who your user base (your customers) will even be.