Disciplined Agile Applied

I recently wrote an article for PMI's ManageIndia magazine entitled Going Beyond Remote Agile: Are You Ready For Your Next Change? In the article I argue that coronavirus disease (COVID-19) is a “black swan event” that has forced most organizations to scramble to figure out how to do their work remotely. Many agile teams are struggling to work remotely, this being particularly tough for some agilists who had mistakenly convinced themselves that they needed to be co-located to be agile. Most teams now are well on the way to adopting common solutions to this challenge. 

But what about next time? I work through five important points to lead you to organize your work smoothly and what you can do the next time you need to identify a new way of working (WoW):

1. Recognize that remote agile isn't new. Others have already dealt with the challenges that you face.
2. Geographic distribution is only one of several scaling factors that you face. You may need to address other, even harder, challenges in the future. 
3. Choice is good. If you know what options are available to you, it's easier to choose your WoW to meet the context that you face.
4. You don't need to work everything out on your own. The Disciplined Agile (DA) toolkit provides consumable, proven, agnostic, and pragmatic options.
5. We need to do better than "fail fast." You can fail a lot less often with a bit of help that we call guided continuous improvement (GCI).

We live in interesting times. The article Going Beyond Remote Agile: Are You Ready For Your Next Change? goes into detail and should provide valuable insights. You can download the full issue of PMI's ManageIndia April 2020 as a pdf.

A common practice is to present estimates in terms of ranges, such as $500K to $900K, rather than point-specific scalars such $650K.  The size of the range that you provide reflects the quality of your understanding of what you’re estimating. PMI’s Practice Standard for Project Estimating (2^nd Edition) suggests an initial guesstimate of -25% to +75%, in general, on projects. We want to present our estimates in ranges so as to set realistic expectations with our stakeholders. 

Your mileage may vary, sometimes dramatically. Barry Boehm, one of the most respected researchers in the software engineering world, has found that a range of -75% to +300% occurs in practice on software projects (see Software Engineering: Barry W. Boehm's Lifetime Contributions to Software Development, Management, and Research).  The initial range is so large because our stakeholders often don’t know what they want, or can’t agree on what they want, and thus it makes it very difficult to estimate.  As Boehm shows this problem is particularly acute in the software world due to the flexibility of software, expectations around software, and the unfortunate tendency of software professionals to agree to requested changes (and then delivering on those promises in sometimes questionable manners).  Marc Bastien presents an insightful exploration of these challenges in his recent book Understanding the Corporate IT Strategy Game.

Figure 1 is based on an actual software development project taking a “predictive” serial approach where the original estimate of 10 months was based on an up-front function point (FP) count.  This is depicted by the first line in Figure 1, as a point-specific scalar estimate.  But as we know estimates are probability distributions and should therefore be presented as a range.  The second line in Figure 1 shows the initial estimate as a probability distribution.  Although it appears to have a large range, which can be disconcerting to anyone hoping for a scalar estimate, this team was able to tighten the range to be 9 to 13 months due to the significant investment they made in function point counting. Over time, as our understanding improves and the uncertainty shrinks the range of the estimate also shrinks.  

Figure 1. Ranged estimates will narrow over time on a healthy project.

An interesting aspect of Figure 1 is that after 3 months you can see that the range of the estimate has tightened a bit and has shifted to the right (the project is likely to take longer than originally believed).  Notice how I said likely to take longer and not going to take longer – that’s because estimates are probability distributions and not certitudes, so I choose to be careful with my language and not imply that the estimate is precise.  Similarly, at the six month point we’re seeing a much narrower range, that’s because we have a much better understanding of both the problem we’re hoping to solve and the solution that we’re implementing to do so.  Figure 1 indicates that the likely project delivery date is clearly slipping over time, with a peak of about 10 months, 13.5 months, and 14.5 months respectively.  Needless to say the project manager had a few testy conversations with the project stakeholders throughout this period.  In the end this team delivered at the 16 month point and then had a second release 6 months later to address quality problems and dropped scope as the result of the “rush job” required to release after 16 months.  So the 10 month project took 22 months in practice. 

Tracking Ranged Estimates Over Time

An estimation trend chart, see Figure 2, is a useful way to depict ranged estimates over time.  This diagram corresponds to what we saw in Figure 1, albeit with monthly rather than quarterly estimates.  The vertical bars represent the ranged estimate each month and the dashed lines are the smoothed trend lines for high and low ends of the ranges. These two lines indicate what is called the “cone of uncertainty,” an indication of relative risk surrounding the precision of your estimates over time.  On a healthy project the two lines will converge at the end of the project, on an unhealthy project that is trending towards failure they will not converge (more on this in a minute).

Figure 2. An estimation trend chart.

At the start of a project you will very likely begin with a guesstimate, which is an estimate with a very wide range.  Over time the range narrows and your guesstimates evolve into estimates.  The point at which guesstimates are considered estimates is up to your organizational culture.  In the case of the project depicted in Figure 2, I suspect that was around the fourth or more likely fifth month of the timeline.

I would be remiss if I didn’t point out that if your estimation range isn’t shrinking over time then that is a sign that your project is in trouble. There are several potential causes for this to happen, including a lack of agreement amongst stakeholders as to the scope of the effort, an inappropriate solution architecture/design strategy, or challenges around how your team is formed (typically you don’t have the right people or sufficient people to get the job done).  In Disciplined Agile (DA) we specifically address these common risks via the stakeholder agreement milestone, the proven architecture milestone, and explicit advice about forming teams respectively. In short, on healthy projects your ranged estimates will tighten over time and on unhealthy projects, if you’re being honest in your estimations, they very likely will not.

The fact that estimates should be presented in ranges is a critical concept for both your executives and your stakeholders to accept.  If they don’t understand the implications of this, and that these ranges really represent probability distributions, then they will not be able to govern effectively.  In the next blog in this series I’m going to share a collection of truisms about estimation that I believe you will find interesting.

In most situations it is a good practice to provide an estimate as a range instead of a fixed-point/scalar.  What I mean by this is that instead saying that a project will take ten months to complete is better to say that it will take between nine and thirteen months given our current understanding of the situation.  In theory the strategy of quoting estimates as ranges is great.  In reality we often run into several unfortunate challenges, which we explore in this blog.

The potential challenges that you face with ranged estimates in practice include: 

1. Some people still want fixed-point estimates. Some people want the “certainty” of a fixed-point estimate, rather than the vagueness of a range.  The idea of a ranged estimate, let alone the concept that an estimate is really a probability distribution, simply doesn’t fit into the way that they want the world to work.
2. Estimates are positively skewed probability distributions. We worked through this conceptIn my previous blog, Estimates Are Probability Distributions.  In Figure 1 the green line shows that this is a positively skewed distribution where there’s a much greater chance that a project will be late or over budget rather than being early or under budget.  The implication is that over time, as we update our estimates based on improved information, the estimate is likely to worsen.
3. People often want greater precision than is appropriate.  Even when people are willing to accept ranged estimates they will often ask for a narrower range than is warranted.  For example, PMI’s Practice Standard for Project Estimating (2^nd Edition) suggests that a project in the start-up phase may have a rough order-of-magnitude (ROM) guesstimate of -25% to +75%.  That’s a huge range that’s hard to sell to people.  In Figure 1 the red line depicts how people ask for a narrower range, such (+/- 25% or even +/- 10%) rather than the harsh reality depicted by the green line.
4. Stakeholders often choose to focus on the low end of the range. An unfortunate axiom of project management is that whenever you give a ranged estimate your stakeholders are likely to focus on the bottom of the range.  For example, tell them something will cost between $50 and $100 and they home in on $50.  Worse yet, they’ll start trying to get you to commit to an even lower figure because they think they can motivate you to do better. 
5. The team often chooses to focus on the high end of the range.  When you guesstimate that a project will take between four and ten months to complete they’ll often choose to believe they have ten months.  Experienced team members, having seen many projects go late or over budget, will believe they have even longer.
6. The historical data either isn’t there, isn’t sufficient, isn’t applicable, or isn’t applied properly. Whew! Thanks to Robin Goldsmith of Go Pro Management who wrote an insightful comment on Estimates Are Probability Distributions about this problem.  The challenge is that few organizations have sufficient historical data from which to base estimates and if they do have such data it is often from dissimilar projects.  Even when organizations do have historical information they often misapply it.

Figure 1. The practical realities of ranged estimates.

The good news is that by being aware of these challenges we can choose to overcome them and apply ranged estimates effectively.  One strategy of doing so is to start with a very wide range, effectively a guesstimate, at the beginning of a project and then as our understanding improves we will update to our estimate and very often narrow the range.   The next blog in this series will explore this approach.

Estimation is an important skill required by many professionals. My experience is that there are three distinct levels of how people think of estimates.  My goal is to focus on the third level, estimates as probability distributions, in this blog.

Let’s begin with a quick overview of the three estimation mindset levels:

1. Estimates as point-specific numbers.  For example, a point-specific estimate would be that this is a million-dollar project that will take six months to complete.  In the case both the cost and time estimates are fixed, scalar numbers. This mindset often stems from a desire to have an “exact number” on which to base decisions.  
2. Estimates as ranges. An example of a ranged estimate is that this project will cost between $800,000 and $1.4 million and take between 4 and 10 months to complete.  In fact, PMI’s Practice Standard for Project Estimating (2^nd Edition) suggests that a project in the start-up phase, what Disciplined Agile (DA) calls Inception, may have a rough order-of-magnitude (ROM) guesstimate of -25% to +75%.  On software projects, Barry Boehm has found that -75% to +300% occurs in practice (see Software Engineering: Barry W. Boehm's Lifetime Contributions to Software Development, Management, and Research).  Yikes.  This estimation mindset often stems from discovering that “exact” estimates often prove to be unrealistic in practice and as a result they need to be more flexible in practice.
3. Estimates as probability distributions.  This is an evolution of ranged estimates in that it reflects the observation that there is a chance that a project will come in below budget (or early) and that there is a chance that a project will come in over budget (or late). This is a strategy that I first learned from Murray Cantor, author of Software Leadership, when I worked with him at IBM Rational.  This estimation mindset tends to be held by people with a strong background in mathematics, in particular statistics.

Figure 1 depicts an estimate as a probability distribution.  It shows a point-specific/fixed estimate, such as $1 million, which is aligned with the peak of the curve.  The area under the graph to the left of the fixed estimate point is the chance that the project will come in under budget and the area to the right is the chance that the estimate will come in over budget.  This probability curve would be determined from past history of previous projects within your organization.  The curve in Figure 1 is “positively skewed” in that there is more area to the right of the curve’s peak than to the left.  The implication is that there is a greater chance of the project being over budget than there is of it being under budget.

Figure 1. Estimates as probability distributions.

Let’s work through how you actually apply this knowledge in practice.  Figure 2 shows an estimate distribution for a fictional project where we are quoting estimates in terms of probabilities.  For example, the chance that this project will come in at $900K or less is the area under the graph to the left of that line, in this case it looks like it’s about 8%.  The chance that it will come in at $1M or less looks to be about 22%, $1.1M or less about 45%, and $1.2M or less about 65%.  The implication is that depending on the level of certainty your stakeholders require you can use this graph to provide an estimate that meets their need.  If your stakeholders wanted 95% surety, for example, then you’d find the point on the estimate distribution where 95% of the area is to the left of the line, in this case it looks like it would be around $1.5 or $1.6 million.  This would be a lot easier, of course, if you were using software to come up with these sorts of numbers rather simply "eye-balling" a graph.  Once again, all of these numbers are based on the past history of previous projects within your organization.

Figure 2. Quoting estimates as probabilities.

There are several challenges with the concept that estimates are probability distributions:

1. Many people don’t understand the concept of probability.  For example, I just opened the weather app on my phone and it says that there is a 70% of snow this hour.  So, if it doesn’t snow in the next hour is the app right or wrong?  It’s right, because there was a 30% chance that it wouldn’t snow.  But what if the app said there was a 90% chance that it will snow, but then it doesn’t?  The app was still right.  The point is that many people will hear a probability such as 70% or 90% and assume that it’s a certainty (100%).  Conversely, they’ll hear that there is a low probability of an event and then assume that it won’t happen and won’t be prepared if it does. 
2. People who desire “predictability” don’t like this idea.  There are still people who prefer the “certainty” of a point-specific estimate, or at least an estimate with a very tight range.  While these expectations might be reasonable on straightforward projects they certainly aren’t on projects where stakeholder needs or implementation strategies evolve constantly (as is typical on software projects, for example).
3. Estimates don’t follow a normal bell curve distribution.  As I indicated earlier, estimates tend to follow a positively skewed distribution.  This is the result of overly-optimistic estimation or pressure to produce an estimate that is palatable to decision makers.  This positive skewing can be frustrating for many people because the implication is that you’re more likely to be over budget or late than you are to be under budget or early respectfully.  

In the next blog in this series we’ll work through the strategy of ranged estimates in greater detail.  

I'm proud to say that I will be giving the opening keynote speech at Agile Middle East (ME) 2020 on March 11th in Dubai. My title of my keynote is #NoFrameworks: How We Can Take Agile Back! which is a reprise of my keynote from the XP 2019 conference last May in Montreal. 

A fundamental philosophy from the early days of Agile is that teams should own their process. Today we would say that they should be allowed, and better yet, enabled, to choose their own way of working (WoW).

This was a powerful vision, but it was quickly abandoned to make way for the Agile certification gold rush. Why do the hard work of learning your craft, of improving your WoW via experimentation and learning, when you can instead become a certified master of an agile method in two days or a program consultant of a scaling framework in four? It sounds great, and certainly is great for anyone collecting the money, but 19 years after the signing of the Agile Manifesto as an industry we’re nowhere near reaching Agile’s promise. Nowhere near it.

Agile had it right in the very beginning, and the lean community had it right all along – teams need to own their process, they must be enabled to choose their WoW. To do this we need to stop looking for easy answers, we must reject the simplistic solutions that the agile industrial complex wants to sell us, and most importantly recognize that we need #NoFrameworks.

As you can see from the conference schedule there is a great line up of speakers. To help support the conference Project Management Institute (PMI) has gotten the organizers to agree to a 20% discount when you use the code "PMI" in all caps when you register. I'm looking forward to the event and if you're in the area I hope you will choose to attend the conference.