Waiting For The Service To End

 

One of the definitions of faith is “firm belief in something for which there is no proof.”[i] It’s a well-known fact that many people, including project managers, believe that modern risk management techniques are a valuable addition to the information streams available to them. But is there any proof of this?

I wasn’t always a risk management skeptic, you know. In fact, at one time I was such a true believer that I wrote some software that pulled project data from a popular critical-path methodology software and performed a single-tiered decision-tree analysis of a project’s WBS (at the reporting level) in order to calculate the contingency budget. Its core formula looked like this:

Cb = ( ∑ (TA1B * OOTA1) + (TA2B * OOTA2) + (TAnB * OOTAIn) ) - PMB

Where Cb is the contingency budget amount, PMB is the Performance Measurement Baseline, TA1B is task alternative #1’s budget, OO is the associated odds of occurrence, TA2B is task alternative #2’s budget, etc. The task alternative’s budgets were estimates of the impacts of various things that could happen to that particular activity. Take their sigma, subtract out the original cost baseline, and you have a risk—based estimate of the amount that should be set aside for contingency.

Of course, to make it usable we had to auto-insert some assumptions. Since 68% of all data points fall within one standard deviation of the mean, we assigned those odds to the tasks’ original estimate. The most likely alternative outcome was given the nominal odds of occurrence of 27%, since that was the amount covered under the next standard deviation.  The fairly unlikely scenarios (worst case scenarios) got the next 4.7%, and the extreme outliers were – again, as preliminary place holders – assigned the last 0.3%. When interviewing the Control Account Managers to collect their insights as to the nature of alternative task outcomes and their impacts, we would always point out that the odds initially assigned were just boilerplate, and that they should feel free to alter them, based on their expertise and experience. Interestingly enough, they rarely did.

Something else really interesting happened during these data collection interviews. When asked about alternative outcomes to the planned tasks, the CAMs were clearly engaging their imaginations, and could admittedly only speculate as to the cost or duration of the alternative endings coming about. For those risk management fans who believe that Monte Carlo analysis provides a more robust (or even valid!) approach, something very similar happens here, too. Once the “most likely,” “best case,” and “worst case” scenarios are established, almost all Monte Carlo software packages for project management invite the analyst to select a distribution curve – usually a leaning bell – to serve as the bounding parameters for the “randomly-generated” alternatives’ cost and schedule impact. But then the exact same type of processing proceeds, just with however many “randomly-generated” additional data points.

To state the blindingly obvious, this is not management science. It is an invitation to speculation, tripped out in probability-and-statistics jargon. These speculations are not, repeat not based on hypothesis, testing, and validation or refutation. Instead, they are based on the projections of the various subject matter experts, or control account managers – in other words, they are largely faith-based, predicated on the idea that their current scope is sufficiently analogous to their previous work to generate an informed guess. Note also that most risk managers will insist that the risk analysis should continue throughout the projects’ life-cycle. Since alternatives that could impact the project are consistently popping up, this is not surprising; but it does point to a specific lack of finality that’s absent from the projects’ baselines. Scope, Cost, and Schedule, at some point, are all “frozen,” or, to one degree or another, contractually binding. Not risk, no siree. It’s almost as if contemporary risk analysis is more about dealing with an unquantifiable future than it is about processing actual data into usable information, the latter being an instantly recognizable aspect of management science, the former more in line with, oh, I don’t know, faith?

As long as I’ve gone this far, I may as well state the obvious conclusion: any system of beliefs that is largely faith-based is NOT management science, and far more closely resembles a religion. I’ve been hearing the faith-based message of the risk managers for decades now, and I’m more than a little sick of it. I’m ready for this service to end.

[i] Merriam-Webster on-line, http://www.merriam-webster.com/dictionary/faith, retrieved August 13, 2016, 14:18 MDT.