When Does Following a Traditional/Serial Lifecycle Make Sense?
| We’re often asked by people when it makes sense for a team to take a traditional approach to software development, and more generally when it makes sense on any type of project. While the focus of this article is on software development projects it also discusses non-software projects too. Sometimes people are honestly trying to identify when each potential lifecycle, including the traditional lifecycle, makes sense to follow. In many cases this request is coming from people who are desperate to continue working in the same old way that they’re comfortable with. They often hope that the context of the situation that they’re in puts them in a position where they don’t need to adopt new ways of working. Some people get “lucky”, although how lucky it is to forgo an opportunity to gain new skills that are currently in demand is questionable at best, but most find that they need to join their colleagues in adopting agile. Traditional development was prevalent in the 70s through 90s, but starting in the mid-90s organizations started to adopt iterative approaches such as the Unified Process (UP) as well as agile approaches such as Scrum or Extreme Programming (XP). Having said all this, the majority of organizations still have a few teams following traditional lifecycles and will often have people who believe that traditional software development is a good idea. And, as I argue in this blog, in a few situations they’re right about that. This blog explores several issues surrounding the traditional approach:
1. What is Traditional Development?Traditional development - also called serial, waterfall, linear, or "predictive" - is based on the idea that the delivery lifecycle should be organized into phases or stages. Figure 1 depicts a classic waterfall lifecycle, similar to what Winston Royce originally proposed (and recommended against) in his 1970 paper. In a pure waterfall the flow is unidirectional (from Requirements to Architecture to…) whereas Figure 1 depicts a “modified waterfall” using bi-directional flow that indicates feedback from a phase may go back to the previous phase. Figure 1. The Waterfall lifecycle.
Figure 2 depicts a gated waterfall where you need to pass through a “quality gate” to move between phases. The “quality gate” tends to be based on the review and acceptance of artifacts – for example a Software Requirements Specification (SRS) coming out of Requirements, a Software Architecture Design (SAD) coming out of Architecture, and so on. It also shows how feedback can go back to any phase via a Change Control process, which on most traditional teams proves to be a change prevention process in practice. I used quotes around “quality gate” because “quality gates” tend to have very little to do with quality in practice and have a lot to do with promoting questionable bureaucracy. My point is that traditional rhetoric may be blinding you to serious deficiencies in this approach. Figure 2. The Gated Waterfall lifecycle.
Figure 3 depicts the V-model version of the traditional approach where the test phase is depicted as multiple stages and there are clear indications of what each testing activity validates (for example, Integration Test validates your architecture). Figure 3. The V-model lifecycle.
The governance strategy with a traditional approach tends to be artifact based (i.e. someone reviews and signs off on your architecture document) rather than risk based (i.e. we proved that the architecture works by building a vertical slice of the solution that implements high-risk requirements). Each phase tends to be staffed with specialists (i.e. requirements are captured by requirements analysts, the design is created by designers, and so on) which tends to motivate a quality gate approach, the development and maintenance of traceability information, and significant management effort to coordinate everything. This in turn increases cost, overall cycle time and decreases ability to support changing requirements (traditionalists typically fear “scope creep” whereas agilists embrace evolving requirements) which in turn decreases stakeholder satisfaction with the end product.
2. When Does Traditional/Waterfall Make Sense?There are several situations when the traditional approach makes sense:
3. What Factors Have No Bearing on When Traditional Makes Sense?My experience is that the following issues do not have an impact, or when they do have very little impact, on the deciding if a traditional strategy is appropriate for your team:
4. But Isn’t Traditional Better at Scale?Not really. Agile and lean are as good or better when applied by large teams, geographically distributed teams, teams in regulatory situations, teams facing domain complexity, teams facing technical complexity, and teams facing organizational distribution. Figure 4 depicts a radar chart of potential tactical scaling factors. Figure 4. Scaling factors faced by software development teams.
So how does traditional fair in comparison with agile and lean strategies in practice? If you poke around at the IT Surveys page you’ll see that the 2014 Software Development at Scale study found that agile teams outperformed non-agile teams across a range of success factors. Similar results can also be found in the 2006, 2008, 2010, 2011, and 2013 Project Success Surveys. The Dr. Dobb’s Journal 2008 Project Success study found that when comparing agile and traditional approaches based on level of geographic distribution that agile teams were at least as successful and often more so, on average, than traditional teams. In other words, agile was less risky. In 2006 DDJ found that agile was as good or better than traditional regardless of team size, but unfortunately I can no longer find those results online. To be clear, some of these surveys are a bit long in the tooth. Having said that, in the past few years organizations have figured out how to successfully apply agile and lean approaches at scale. I suspect that agile and lean will both have pulled ahead even further compared with traditional at scale. We’re actively looking into these sorts of issues so stayed tuned to this blog for future research results. And, we’re not the only ones who are getting these sorts of results. Go poking around on the web and find out for yourself.
5. Does Disciplined Agile (DA) Support the Traditional Lifecycle?The DA toolkit does not yet support a traditional lifecycle, although an upcoming release in Q2 2020 will bring some aspects of serial/traditional into play. Having said that, DA has always recognized that in the vast majority of organizations you will have a multi-modal approach where some teams are following an agile approach, some are lean, some are taking a continuous delivery approach, and some are still following traditional. The more disciplined your organization, the more skilled your people, the less likely it is to have traditional teams in practice.
6. Doesn’t DA Support a Serial Lifecycle?Yes, but it’s risk-based, not artifact nor activity based as you see with traditional approaches. The two project lifecycles supported by DAD, the Scrum-based Agile lifecycle and the Kanban-based Lean lifecycle, have three phases: Inception, Construction, and Transition. These phases, overviewed in Figure 5, capture the ideas that a project team needs to be initiated somehow (Inception), the solution must be developed (Construction), and then released into Production (Transition). Figure 5. The phases and milestones of DAD project teams.
When you have stable (long-lived) product teams Inception tends to disappear (with the exception of the need for occasional modeling and planning sessions to think things through) and Transition evolves from a multi-day or multi-week phase into a multi-minute or multi-hour activity. Furthermore, the lifecycle evolves over time as you improve, moving from a phased agile/lean project lifecycle into a continuous delivery lifecycle. More on this in future blogs. 7. What About Non-Software Projects?Although much of the advice presented above applies to non-software projects there are some differences in the non-software world. For example, in the physical construction world (think building buildings, oil refineries, highways, ...) they will often work in a traditional manner. This happens for many good reasons:
Having said that, even when it makes sense to follow a more traditional approach you can still benefit from applying agile strategies on your project. For example I was recently involved in a physical construction project and I was struck by how much of the actual work was performed in an agile manner. At a high-level it looked like a traditional/serial project, and it was, but on a day-to-day basis it was reasonably agile within the limits they had. So it was more of a hybrid strategy masquerading as traditional. 8. Why Should You Listen to Me?Here is a brief description of my background in traditional software development:
The point is that I believe I have sufficient experience with traditional approaches to speak about where it does and doesn’t work. I hope this blog has provided some valuable insights for you.
|
Right-sizing your agile process? Start in the Middle
| Is your organization concerned with the cost and time required to adopt agile strategies? Are you just starting out with agile and hoping to improve your chances of success by learning from the experiences of those who have gone before you? Are you part way into your agile transformation but struggling to figure out how it all fits together? If you answered yes to any of these questions please read on. In this blog posting we discuss what it means to “right size” your software process to meet the needs of the unique situation that your team finds itself in. We discuss two common anti-patterns: starting with a process framework that is much too large for your needs and starting with one that is far too small for your needs. We argue that it’s much better to avoid these extremes and instead take a middle-ground approach by starting with a toolkit that is much closer to your actual needs. Extreme #1: Large process repositoryThe first process right-sizing anti-pattern is to start with a large process repository, the classic example being IBM’s Rational Unified Process (RUP). Although RUP is much maligned within the agile community the fact is that if you were to examine it with an open mind that there are many very good ideas promoted by RUP. Be that as it may. The basic strategy with RUP is that you need to tailor down the process, often dramatically, to meet the unique needs of the situation your team finds itself in. To do this requires significant process expertise, time, and money. Extreme #2: Small methodologyAt the other extreme is the right-sizing anti-pattern of starting with a small methodology, the classic example in this case being Scrum. Although there is significant support for Scrum within the agile community, or at least among people who are new to agile, we’ve been seeing for a long time now that organizations are also running into trouble with this approach too. With Scrum the idea is that you add in the techniques that Scrum doesn’t address to right-size your process. Because Scrum proves to be only a very small part of the overall picture, to do this requires significant process expertise, time, and money. A few years ago Forrester Research discovered that the majority of organizations “doing Scrum” had actually tailored it into what they called Water-Scrum-Fall (others call this Scrumifall). As we describe in Going Beyond Scrum this occurs for several reasons. First, Scrum doesn’t address the full delivery lifecycle, instead choosing to focus on the construction portion of it. As a result organizations tend to stick with what they know, a heavy project initiation phase and a heavy solution deployment phase. Second, Scrum only addresses only a small portion of what you need and explicitly leaves technical issues up to you. These issues include topics such testing, programming, architecture, governance, documentation, deployment and many others. As a result teams are left to piece together a process strategy that works for them, at the very same time that they’re struggling to understand the fundamentals of agile and lean software development. They rarely have the process expertise to do that and as a result end up having to hire hordes of expensive Scrum coaches, few of whom seem to understand the enterprise-class realities that your teams actually face. This is a risky and expensive proposition indeed. The Effective Middle GroundBoth of these anti-patterns represent extremes: start with something large and cut it down to size, or start with something small and build it up to meet your needs. Why not start with something much closer to what you actually need? Doesn’t that make a lot more sense? Why do you need to do all this process work? Because someone wants to sell you tooling? Because someone else wants to sell you expensive coaching and questionable certification strategies? Isn’t it time to consider a more pragmatic strategy? The Disciplined Agile (DA) toolkit is a more effective middle ground. It addresses the full delivery lifecycle, as does RUP (but not Scrum), and even gives you several choices from which to select. Sometimes a Scrum-based lifecycle is appropriate, sometimes a Lean lifecycle is, other times a continuous delivery lifecycle is best, and sometimes an exploratory “lean start up” lifecycle is. Different teams, different situations. DAD starts with a lightweight approach, as does Scrum but not RUP, helping you to avoid the bloatware of RUP and filling in the numerous blanks left by Scrum. DAD also gives you lightweight tailoring advice in the form of process goal diagrams, in many ways a cross between mind-maps and decision trees, that make your process choices explicit. The RUP process tailoring advice, if you bother to read it at all, is rather heavy handed and the Scrum tailoring advice boils down to “you’re smart, you can figure it out and if your run into trouble hire a coach.” Isn’t it time to abandon the extremes?
In SummaryThe following infographic summarizes the main points in this blog posting. |
Right-sizing your agile process? Start in the Middle
| Is your organization concerned with the cost and time required to adopt agile strategies? Are you just starting out with agile and hoping to improve your chances of success by learning from the experiences of those who have gone before you? Are you part way into your agile transformation but struggling to figure out how it all fits together? If you answered yes to any of these questions please read on. In this blog posting we discuss what it means to “right size” your software process to meet the needs of the unique situation that your team finds itself in. We discuss two common anti-patterns: starting with a process framework that is much too large for your needs and starting with one that is far too small for your needs. We argue that it’s much better to avoid these extremes and instead take a middle-ground approach by starting with a toolkit that is much closer to your actual needs. Extreme #1: Large process repositoryThe first process right-sizing anti-pattern is to start with a large process repository, the classic example being IBM’s Rational Unified Process (RUP). Although RUP is much maligned within the agile community the fact is that if you were to examine it with an open mind that there are many very good ideas promoted by RUP. Be that as it may. The basic strategy with RUP is that you need to tailor down the process, often dramatically, to meet the unique needs of the situation your team finds itself in. To do this requires significant process expertise, time, and money. Extreme #2: Small methodologyAt the other extreme is the right-sizing anti-pattern of starting with a small methodology, the classic example in this case being Scrum. Although there is significant support for Scrum within the agile community, or at least among people who are new to agile, we’ve been seeing for a long time now that organizations are also running into trouble with this approach too. With Scrum the idea is that you add in the techniques that Scrum doesn’t address to right-size your process. Because Scrum proves to be only a very small part of the overall picture, to do this requires significant process expertise, time, and money. A few years ago Forrester Research discovered that the majority of organizations “doing Scrum” had actually tailored it into what they called Water-Scrum-Fall (others call this Scrumifall). As we describe in Going Beyond Scrum this occurs for several reasons. First, Scrum doesn’t address the full delivery lifecycle, instead choosing to focus on the construction portion of it. As a result organizations tend to stick with what they know, a heavy project initiation phase and a heavy solution deployment phase. Second, Scrum only addresses only a small portion of what you need and explicitly leaves technical issues up to you. These issues include topics such testing, programming, architecture, governance, documentation, deployment and many others. As a result teams are left to piece together a process strategy that works for them, at the very same time that they’re struggling to understand the fundamentals of agile and lean software development. They rarely have the process expertise to do that and as a result end up having to hire hordes of expensive Scrum coaches, few of whom seem to understand the enterprise-class realities that your teams actually face. This is a risky and expensive proposition indeed. The Effective Middle GroundBoth of these anti-patterns represent extremes: start with something large and cut it down to size, or start with something small and build it up to meet your needs. Why not start with something much closer to what you actually need? Doesn’t that make a lot more sense? Why do you need to do all this process work? Because someone wants to sell you tooling? Because someone else wants to sell you expensive coaching and questionable certification strategies? Isn’t it time to consider a more pragmatic strategy? The Disciplined Agile (DA) toolkit is a more effective middle ground. It addresses the full delivery lifecycle, as does RUP (but not Scrum), and even gives you several choices from which to select. Sometimes a Scrum-based lifecycle is appropriate, sometimes a Lean lifecycle is, other times a continuous delivery lifecycle is best, and sometimes an exploratory “lean start up” lifecycle is. Different teams, different situations. DAD starts with a lightweight approach, as does Scrum but not RUP, helping you to avoid the bloatware of RUP and filling in the numerous blanks left by Scrum. DAD also gives you lightweight tailoring advice in the form of process goal diagrams, in many ways a cross between mind-maps and decision trees, that make your process choices explicit. The RUP process tailoring advice, if you bother to read it at all, is rather heavy handed and the Scrum tailoring advice boils down to “you’re smart, you can figure it out and if your run into trouble hire a coach.” Isn’t it time to abandon the extremes?
In SummaryThe following infographic summarizes the main points in this blog posting. |
Comparing DAD to the Rational Unified Process (RUP) - Part 1
Categories:
DAD discussions,
agile,
DAD,
mark lines,
disciplined agile delivery,
rational unified process,
RUP,
Scrum
Categories: DAD discussions, agile, DAD, mark lines, disciplined agile delivery, rational unified process, RUP, Scrum
|
Last week I was discussing DAD with a new client and he asked me “Is DAD just an Agile version of RUP?” In a word, no. DAD is a toolkit composed of a hybrid of methods and practices as shown in the diagram. It includes the best of Scrum, Extreme Programming (XP), Agile data and modeling, and yes, the Unified Process (UP). DAD also includes additional content such as agile governance that is not present in any of these methods. As the diagram indicates, probably the method that adds most to DAD is XP, not the UP. DAD does not prescribe a use case-driven approach, or insist that OOAD be rigorously applied to build out services/components. A use case-driven approach is a potential practice to apply but there is a danger that this could lead to an exhaustive requirements specification which is not particularly agile. We would prefer to use a user story-driven approach if that makes sense within the context of your project. User stories might not be the right choice either. Perhaps you are in a regulatory environment that demands a traditional software requirements specification (SRS). The key point is that you will have to adapt to the situation that you find yourself in. This is why we prioritize the team’s work with a work item list comprised of work items, rather than Scrum’s backlog comprised of user stories. Using a work item list allows us the flexibility to put any type of work onto our backlog, extending the applicability of DAD to many types of projects beyond those for which RUP or Scrum would be ideally suited. DAD is goal-driven, not artifact-driven. It does not prescribe practices or specific artifacts. Rather, it suggests alternative strategies than can be applied at certain parts of the lifecycle with the pros and cons for each, but which ones you choose is up to you. In my next post I will describe which aspects of the Unified Process did make it into DAD and why. |












