In most situations it is a good practice to provide an estimate as a range instead of a fixed-point/scalar. What I mean by this is that instead saying that a project will take ten months to complete is better to say that it will take between nine and thirteen months given our current understanding of the situation. In theory the strategy of quoting estimates as ranges is great. In reality we often run into several unfortunate challenges, which we explore in this blog.
The potential challenges that you face with ranged estimates in practice include:
Some people still want fixed-point estimates. Some people want the “certainty” of a fixed-point estimate, rather than the vagueness of a range. The idea of a ranged estimate, let alone the concept that an estimate is really a probability distribution, simply doesn’t fit into the way that they want the world to work.
Estimates are positively skewed probability distributions. We worked through this conceptIn my previous blog, Estimates Are Probability Distributions. In Figure 1 the green line shows that this is a positively skewed distribution where there’s a much greater chance that a project will be late or over budget rather than being early or under budget. The implication is that over time, as we update our estimates based on improved information, the estimate is likely to worsen.
People often want greater precision than is appropriate. Even when people are willing to accept ranged estimates they will often ask for a narrower range than is warranted. For example, PMI’s Practice Standard for Project Estimating (2nd Edition) suggests that a project in the start-up phase may have a rough order-of-magnitude (ROM) guesstimate of -25% to +75%. That’s a huge range that’s hard to sell to people. In Figure 1 the red line depicts how people ask for a narrower range, such (+/- 25% or even +/- 10%) rather than the harsh reality depicted by the green line.
Stakeholders often choose to focus on the low end of the range. An unfortunate axiom of project management is that whenever you give a ranged estimate your stakeholders are likely to focus on the bottom of the range. For example, tell them something will cost between $50 and $100 and they home in on $50. Worse yet, they’ll start trying to get you to commit to an even lower figure because they think they can motivate you to do better.
The team often chooses to focus on the high end of the range. When you guesstimate that a project will take between four and ten months to complete they’ll often choose to believe they have ten months. Experienced team members, having seen many projects go late or over budget, will believe they have even longer.
The historical data either isn’t there, isn’t sufficient, isn’t applicable, or isn’t applied properly. Whew! Thanks to Robin Goldsmith of Go Pro Management who wrote an insightful comment on Estimates Are Probability Distributions about this problem. The challenge is that few organizations have sufficient historical data from which to base estimates and if they do have such data it is often from dissimilar projects. Even when organizations do have historical information they often misapply it.
Figure 1. The practical realities of ranged estimates.
The good news is that by being aware of these challenges we can choose to overcome them and apply ranged estimates effectively. One strategy of doing so is to start with a very wide range, effectively a guesstimate, at the beginning of a project and then as our understanding improves we will update to our estimate and very often narrow the range. The next blog in this series will explore this approach.
Barry Boehm's Cone of Uncertainty is one method of providing a ranged estimate. A Monte Carlo simulation can also give you a range of possible outcomes to help understand one's confidence in achieving a specific target.
Interestingly, I promote the usage of some Monte-Carlo for estimation, more realistic. And the Cone of Uncertainty from Barry Boeh is great at explaining why Initial estimate gets more precise with time, better definition of the requirements.
Great analogy and points. I totally agree with you as I see this with our teams and stakeholders.
However, on the other hand, under Point 2, you mentioned the following: “ as we update our estimates based on improved information, the estimate is likely to worsen.”
Unless I am missing your point, I am not sure I totally agree with this. In our projects, as more information is available and the design in refined, we refine our estimates and they usually are more precise. Can you elaborate on this point please ?
There is another approach is using an abstract unit like story point in software team. It doesn't explicitly refer how long it will take. & can imply a range when we forecast or plan
Kiron& Vincent, in the next blog in this series I'll cover cone of uncertainty (it's rarely a true cone in practice but close enough). Hadn't thought of covering monte carlo as I've found very few customers are that sophisticated. But it's definitely worth a paragraph or two.
Rami, I wasn't clear. The next blog works through an actual example. What I mean is that although the range typically shrinks (hence the quality of it improves) at the same time the date/cost often slips (so the numbers themselves get worse from a stakeholder point of view). Like I said, example to come.
Anh, yes, abstract units like story points are one of a multitude of techniques that can be used for projecting cost and schedules.
It is interesting that positively skewed distribution are bought by stakeholders as that is what they want to see.
Eager to see your next blog on this issue.
Interesting. Many forget the meaning of the word 'estimate' and take it as commitment. Any estimate must be accompanied by the degree of confidence in it or (better) be expressed as a range, between x and y. in my experience the famous cone of uncertainty is no longer valid in Agile. At least the convergence to zero doesn't make any sense. The backlog is not cleared up and if it is that's certainly not Agile.
A good argument for keeping the good side of Lean Six Sigma in Agile.
If you have a project team then the backlog of what you're going to deliver for that project can in fact converge to zero although isn't guaranteed because stuff happens.
For a dedicated product team, if you working in terms of regular releases, then it's the same sort of thing. In some ways you can consider a release to be a project for that team (potentially dangerous thinking, but it happens).
For a dedicated team that's taking more of a continuous delivery approach where we release frequently then estimating doesn't make a lot of sense any more. This is because of two things: First, the costs are easy to calculate as it's the charge out rate of the team. Second, when you release frequently, particularly weekly or better, people stop asking for estimates because they can see what they're getting for their investment.
@Scott, all good for software development only but a project is more than that. Everything is a project. In continuous delivery you need to estimate cost/benefit. Salaries must come from somewhere. watch the General Magic documentary.
