Pathologies In Our Decision-Making

 

The title of this piece is a phrase turned by Nassim Taleb, in his best-selling book The Black Swan, The Impact of the HIGHLY IMPROBABLE (Random House, 2007). He discusses several such pathologies, but his strongest (in my opinion) arguments and condemnations are saved for managers and analysts who rely on the fields of probability and statistics. As I relay in my recently-released, must-have book, Game Theory in Management, Taleb goes so far as to say (on page 355 of the paperback version) “This proves that everything relying on ‘standard derivative,’ ‘variance,’ ‘least square derivation,’ etc. is bogus.”

Proceeding from his very well-argued points, I would like to draw the conclusion that virtually all of what passes for modern Risk Management theory is invalid, and ought to be abandoned.

Gantthead wanted its bloggers to take on Olympic-sized issues in July, and Risk Management is pretty darn huge. A Google search of “Risk Management Consultants” returned over 90M hits on July 14, 2012. Risk Management is one of nine chapters in PMI®’s hallowed Guide to the Project Management Body of Knowledge®. There’s even an ISO Standard for it.

To be clear, risk management does have a place in project management, but it’s much smaller than advertised.  Analysis methods Decision Tree and Monte Carlo simulation can provide a reasonable estimating parameter for calculating how much in funds reserve a given project should have, or identify appropriate targets for insuring risks. Past that, RM’s claims are pretty much overblown, and the axiomatic “80% confidence interval” is right out.

Consider the Drake Equation, often invoked to defend (and attract funds for) that fool’s errand, the Search for Extra-Terrestrial Life, or SETI. The formula looks like this:

N = N * fp ne fl fi fc fL

Where N is the number of stars in the Milky Way galaxy; fp is the fraction with planets; ne is the number of planets per star capable of supporting life; fl is the fraction of planets where life evolves; fi is the fraction where intelligent life evolves; fc is the fraction that communicates; and fL is the fraction of the plant’s life during which the communicating civilizations live. As Michael Crichton pointed out in his lecture “Aliens Cause Global Warming” (Caltech Michelin Lecture, January 17, 2003) nobody has any idea what any of these parameters might be. Even the first one – the number of stars in the Milky Way – could be anywhere in between 10 billion and 40 billion. For truly experienced risk managers, that’s a range of thirty billion, and that’s the parameter we have the best shot at knowing! The Drake Equation is nothing more than an invitation to speculate, placed in pseudo-scientific terms. As Crichton himself put it, “An equation that could mean anything means nothing.”

Now consider the formula for calculating a contingency fund using a single-tier Decision-Tree analysis:

Cn = BAC – [(E1$ * E1%) + (E2$ * E2%) + (En$ * En%)]

Where Cn is the contingency budget, BAC is the budget at completion, E1$ is the cost of possible event one occurring, E1% is the odds of event one occurring, all the way through event n, which is the last possible event included in the analysis. Where do the existence of (and cost/schedule estimates for) these “events” come from? Usually a risk analyst working with a subject matter expert, or the owner of the particular cost account or work package being analyzed. Do I have to say it? The Decision-Tree analysis, like the Drake Equation, is simply an invitation to speculate, with such speculations almost certainly being chocked-full of the analysts’ cognitive biases.

But you’ll never hear risk managers admitting as such. They love asserting that they can, to a certain degree, quantify the future events in a given project. And, of course, knowledge of the future is pure management gold, making any plausible claim of being able to capture such information extremely attractive. But, just as medieval alchemist tried to combine common materials with lead to create literal gold, risk managers seek to combine common information streams into a reliable narrative of how the future of a project should be expected to unfold. Of course, they can’t possibly, and no amount of statistical jargon can change that fact.

One last little stinger example: after reviewing the techniques in Project and Program RISK MANAGEMENT, A Guide to Managing Project Risks and Opportunities (PMI Publishing, 1976), I estimate that there is a 72% chance that the majority of the techniques in Project and Program RISK MANAGEMENT, A Guide to Managing Project Risks and Opportunities (PMI Publishing, 1976), will be considered as laughably obsolete as medieval alchemy within 45.2 years, with an 80% confidence interval. How did I come up with my variables (such as, how does one evaluate the comparable laughability of misguided medieval pseudo-scientific pursuits)?

Trust me – I’m an expert.