Game Theory in Management

The title of this piece is a phrase turned by Nassim Taleb, in his best-selling book The Black Swan, The Impact of the HIGHLY IMPROBABLE (Random House, 2007). He discusses several such pathologies, but his strongest (in my opinion) arguments and condemnations are saved for managers and analysts who rely on the fields of probability and statistics. As I relay in my recently-released, must-have book, Game Theory in Management, Taleb goes so far as to say (on page 355 of the paperback version) “This proves that everything relying on ‘standard derivative,’ ‘variance,’ ‘least square derivation,’ etc. is bogus.”

Proceeding from his very well-argued points, I would like to draw the conclusion that virtually all of what passes for modern Risk Management theory is invalid, and ought to be abandoned.

Gantthead wanted its bloggers to take on Olympic-sized issues in July, and Risk Management is pretty darn huge. A Google search of “Risk Management Consultants” returned over 90M hits on July 14, 2012. Risk Management is one of nine chapters in PMI®’s hallowed Guide to the Project Management Body of Knowledge®. There’s even an ISO Standard for it.

To be clear, risk management does have a place in project management, but it’s much smaller than advertised.  Analysis methods Decision Tree and Monte Carlo simulation can provide a reasonable estimating parameter for calculating how much in funds reserve a given project should have, or identify appropriate targets for insuring risks. Past that, RM’s claims are pretty much overblown, and the axiomatic “80% confidence interval” is right out.

Consider the Drake Equation, often invoked to defend (and attract funds for) that fool’s errand, the Search for Extra-Terrestrial Life, or SETI. The formula looks like this:

N = N * fp ne fl fi fc fL

Where N is the number of stars in the Milky Way galaxy; fp is the fraction with planets; ne is the number of planets per star capable of supporting life; fl is the fraction of planets where life evolves; fi is the fraction where intelligent life evolves; fc is the fraction that communicates; and fL is the fraction of the plant’s life during which the communicating civilizations live. As Michael Crichton pointed out in his lecture “Aliens Cause Global Warming” (Caltech Michelin Lecture, January 17, 2003) nobody has any idea what any of these parameters might be. Even the first one – the number of stars in the Milky Way – could be anywhere in between 10 billion and 40 billion. For truly experienced risk managers, that’s a range of thirty billion, and that’s the parameter we have the best shot at knowing! The Drake Equation is nothing more than an invitation to speculate, placed in pseudo-scientific terms. As Crichton himself put it, “An equation that could mean anything means nothing.”

Now consider the formula for calculating a contingency fund using a single-tier Decision-Tree analysis:

Cn = BAC – [(E1$ * E1%) + (E2$ * E2%) + (En$ * En%)]

Where Cn is the contingency budget, BAC is the budget at completion, E1$ is the cost of possible event one occurring, E1% is the odds of event one occurring, all the way through event n, which is the last possible event included in the analysis. Where do the existence of (and cost/schedule estimates for) these “events” come from? Usually a risk analyst working with a subject matter expert, or the owner of the particular cost account or work package being analyzed. Do I have to say it? The Decision-Tree analysis, like the Drake Equation, is simply an invitation to speculate, with such speculations almost certainly being chocked-full of the analysts’ cognitive biases.

But you’ll never hear risk managers admitting as such. They love asserting that they can, to a certain degree, quantify the future events in a given project. And, of course, knowledge of the future is pure management gold, making any plausible claim of being able to capture such information extremely attractive. But, just as medieval alchemist tried to combine common materials with lead to create literal gold, risk managers seek to combine common information streams into a reliable narrative of how the future of a project should be expected to unfold. Of course, they can’t possibly, and no amount of statistical jargon can change that fact.

One last little stinger example: after reviewing the techniques in Project and Program RISK MANAGEMENT, A Guide to Managing Project Risks and Opportunities (PMI Publishing, 1976), I estimate that there is a 72% chance that the majority of the techniques in Project and Program RISK MANAGEMENT, A Guide to Managing Project Risks and Opportunities (PMI Publishing, 1976), will be considered as laughably obsolete as medieval alchemy within 45.2 years, with an 80% confidence interval. How did I come up with my variables (such as, how does one evaluate the comparable laughability of misguided medieval pseudo-scientific pursuits)?

Trust me – I’m an expert.

Yeah, I know this blog’s title sounds counter-intuitive. If “success” is a problem, than what in the world isn’t?

But stay with me a few sentences, and I promise to make things clear. I want to examine the term of a person “being the victim of their own success.” It usually applies to one who has been successfully implementing a certain tactic for so long that they no longer recognize new scenarios where the tactic is inappropriate, and then fail trying to implement this favorite tactic where it doesn’t belong. Put another way, when we enjoy success on a large scale, it’s very easy to attribute such an outcome with tactics, behaviors, people, and environmental factors that really had little (or even nothing) to do with the achievement being enjoyed. 

As Eric Berne wrote about in his classic book Games People Play (Grove Press, 1964), and I build on Dr. Berne’s ideas in my recently-released must-have book, Game Theory in Management, people tend to construct narratives, or scripts in their minds that serve a variety of purposes, including explaining who we are to ourselves and why our pasts have unfolded the way they did. Now consider the project team member or manager who has been associated with a project that, once completed, was considered a significant achievement and on a large scale. Unless such a person has an advanced understanding of the rules of evidence, logic, and causality (and in many cases, even if they do), odds are that significant parts of the narrative they embrace will contain cognitive biases, such as confirmation bias, which “informs” them of the characteristics of that achievement. Sometimes the association will be valid, and loosely-held. Other times, the association is invalid, and tightly-held.

For example, I was once providing the cost and schedule performance information for a very large and high-profile project. The Control Account Manager (or CAM) for the project management task had recently come off of another high-profile project that had come in on-time and under-budget. At the very first meeting on how we would set up the Earned Value and Critical Path systems, this CAM announced that my team’s first priority would be to generate … a “swim lane chart.” As I exchanged puzzled looks with my team, the CAM went to a white board and drew a set of parallel horizontal lines, and then place boxes representing tasks in between the lines. The “lanes” represented organizations within the project team.

“So, what you’re asking for is essentially a PERT chart, segregated by performing organization?” I asked. After he affirmed that that was what he was after, I offered “I think we need to develop a Work Breakdown Structure first, and then the cost and schedule baselines. Then we’ll be able to create your chart.”

The CAM was adamant. The “swim lane chart” had to be our first priority, period, and he would hear no talk of WBSs or baselines. Apparently, he had had access to these org-chart-arranged PERT charts in his earlier project, had become convinced that this particular format represented his information system silver bullet, and no amount of discussing the particulars of his request could dissuade him. Instead, the team had to essentially create the Work Breakdown Structure and cost/schedule baselines in secret, while maintaining publicly that we were exclusively pursuing the data elements to create the “swim lane chart.”

We were lucky, then. Other “successful” project managers aren’t so easy to work around. Their approaches to new problems can become chock-full of formulaic technical approaches, and any perceived deviation simply must be due to a lack of expertise or recalcitrance from the staff, neither of which is tolerable.

Conversely, managers coming off of poorly-performing projects, or even disastrous ones, tend to have greater respect for common but formidable problems that lead to overruns and delays. They also tend to retain that certain level of humility that leads them to at least listen to the other members of the project team, especially when different alternatives to approaching problem-solving are discussed. Of course, none of this is true when projects fail, and their decision-makers are not held responsible; but, in a free market environment, successful managers rarely repeat mistakes, whereas successful projects may have incurred many mistakes, but they were never identified, much less analyzed.

Andthat’s the problem with big success.

In honor of the Olympics beginning later this month, I would like to turn my readers’ attention towards very large project management problems; or, better stated, with management problems of very large projects. There’s just something about very large projects that, when they go South, they do so in dramatic fashion, like the Hindenburg or the Titanic. Interestingly enough, Titanic had a sister ship named Olympic, which distinguished herself by colliding with two different ships in her career, which is, I suppose, a superior fate to either of her sisters (the third ship in the class, Brittanic, was sunk during World War I after striking a mine). One of the ships she hit wasn’t even moving, being a lightship.

So, in examining the whole the-bigger-they-are-the-harder-they-fall motif, I would like to discuss the National Ignition Facility, or NIF. As I state in my just-released must-have book, Game Theory in Management; Modelling Business Decisions and their Consequences (http://www.gowerpublishing.com/isbn/9781409442417 ),

The NIF was a project designed to help researchers answer questions surrounding nuclear fusion. The concept was to surround a pellet of a hydrogen isotope with 192 high-powered lasers that could deliver sufficient energy to the target quickly enough to induce nuclear fusion, the same type of reaction that the Sun uses. The project was spearheaded by a scientist, who swore to Senate Appropriations Committee staffers that if Lawrence Livermore National Laboratory was selected to build the project, he would make sure it stayed on budget. Unfortunately, cascading risk events rendered that promise impossible to keep. Work began in 1997 with an estimated budget of $1.1 billion. The final price was nearly quadruple that amount. The Wikipedia article on NIF states:

The Pulsed Power Conditioning Modules (PCMs) suffered capacitor failures, which in turn caused explosions. This required a redesign of the module to contain the debris, but since the concrete structure of the buildings holding them had already been poured, this left the new modules so tightly packed that there was no way to do maintenance in-place. Yet another redesign followed, this time allowing the modules to be removed from the bays for servicing. Continuing problems of this sort further delayed the operational start of the project, and in September 1999 an updated DOE report stated that NIF would require up to $350 million more and completion would be pushed back to 2006.[1]

More problems like this manifested, but perhaps the most damaging risk event had to do with dust. High-energy lasers do not react well to dust: if the various mirrors and lenses used to aim and manipulate the beams have so much as a speck on them, the damage can be explosive, immediate, and expensive. Insufficient consideration was given to the difficulties of performing an extremely large construction effort, with new technology and large, heavy, and expensive components, all in a clean room environment. The early problems with design, construction, and dust cascaded to the point that NIF became the poster child for the perils of failing to perform the project management function adequately.

The truly ironic aspect to the NIF management disaster was the lateness with which management embraced Earned Value and Critical Path-based management information systems. The original estimate of $1.1 Billion (USD) would turn into $4.2 Billion by the time the construction part of the project was complete. Now, it may well be that had NIF management approached the U.S. Department of Energy with an initial estimate of $4.2 Billion, the DOE may have approved the project anyway. But that’s not what happened, leaving the typical management analyst to wonder what might have been. Even a simple Earned Value Management System would have provided early warning of the upcoming overruns, as well as the delays. But such basic project management information systems weren’t implemented in the early phases of the project. Management felt that they could get a handle on the cost performance of the project without an EVMS, with not-enough-life-boats or let’s-fill-the-dirigible-with-Hydrogen-sized consequences (well, to be honest, nobody died, but $3.1 Billion could have done a lot of life-improving, don’t you think?).

Of course, there’s always the possibility that, had a functioning EVMS been in place early-on at NIF, it would have had a similar impact to project controller Frederick Fleet calling NIF’s PM, and shouting “The Cost Performance Index-based Estimate at Completion is indicating a massive overrun, right ahead!”, and then watching helplessly as the failure unfolded. But, from an information technology point of view, it would have at least given some credence to the idea that management wasn’t flying completely blind. Being at the helm of a large project crash leaves those responsible with only a couple of narratives to further, neither of which is very attractive:

·         The “I didn’t see it coming, but then, nobody could have seen it coming” story line. The problem here is, if this kind of problem is out there, lying in wait for any similar effort, why would anyone undertake that kind of project in the first place?

·         The “I knew it was a possibility, but I thought our response to that event would have alleviated its negative consequences” script. This is easily overturned by noting either the number of lifeboats on board being far fewer than needed, or the fact that the lack of an EVMS in the earliest stages rendered project personnel hopelessly ignorant of performance-based cost overruns.

Of course, hindsight is 20-20, and there is a distinct element of unfairness to criticizing management decisions after-the-fact. I understand that, and do not wish to, as they say in American football, “pile on.” On the other hand, if you are (were) in charge of a large project that incurred massive overruns and delays, and you didn’t have a working Earned Value Management System from the get-go, maybe, just maybe you deserve a bit of being piled-on.

What do you think?

In July, Gantthead writers and bloggers will be addressing very large, Olympic-sized themes, so for this, my last blog of June, I thought I’d get some small things out of the way.

First up is the spelling of the possessive impersonal pronoun. “It’s” is a contraction, of “it is.” And yet, when I look at the comments section of major news stories coming across the internet wire, time and time again I see people writing “it’s” when they really mean “its,” as if we should adjust our view on politics or religion based on the fevered rantings of a person who couldn’t pass seventh grade English. When I’m teaching my company’s Variance Analysis Report Preparation class, I make it a point to go up to the white board, and write the word “theirs.” Then, I call on someone in the class.

“What does that say?”

“Theirs.”

“Is it misspelled?”

“No.”

Then, I’ll write “his,” and then “hers,” and call on two more people, who both affirm neither possessive pronoun is misspelled. Then I’ll write “its.” This will invariably elicit giggles.

“You laugh,” I’ll begin, “but I guaran-dam-tee that if I review ten VARs prepared by this organization, I’ll find this error at least once. And you have no idea how stupid that makes us look.”

Next, I want to address those people who insist that any useful Earned Value analysis must be predicated on an extremely accurate estimate. And, if an extremely accurate estimate is not forthcoming, for whatever reason, then, well, the whole EV system must be abandoned. This is idiocy of Cecil B. DeMille proportions, which means, I suppose, that I’ve crossed the threshold from little things to big things, but stay with me. There is no cost management without Earned Value, period. Yet, there are many who either willfully ignore this assertion, or else have a motive for resisting it (yeah, accountants, I’m talking to you!). In order to avoid having a legitimate, simple EV system overtake their bogus faux-cost management system, they will attempt to push the narrative that EV systems are difficult, bulky, and expensive, none of which is true. Which brings us back to the must-have-very-accurate-baselines BS.

The truth is that extremely valuable management information can be gleaned from simple Earned Value systems, even when the time-phased budget is inaccurate. The traditional quick-and easy Estimate at Completion formula, Budget at Completion (BAC) divided by the Cost Performance Index (CPI), can be reduced algebraically to dividing cumulative actual costs (finally! The accountants are relative!) by the cumulative percent complete (on a task, or overall project). This calculation has been shown to be within 10% on projects or tasks that have surpassed the 15% completion point. Note that this piece of information – which has to be considered extremely valuable to anyone intelligent enough to read this blog  – does not require an accurate original estimate. It doesn’t even require a time-phased budget. Those who insist on excessive effort or accuracy in a project’s original budget are actually working against having a legitimate cost management system in place.

Finally, I want to vent my frustration on those who like to pretend they understand Critical Path theory, but really don’t. The most common and obvious manifestation of these people’s ignorance is when they insist that a certain task within the schedule network must be on the critical path, since it’s so important, don’t you know. Of course, whether or not a specific activity shows up on the critical path has nothing to do with its (notice I used the right one?) priority or visibility. It’s (again!) entirely a function of its (three in a row) duration and schedule logic, making anyone who posits to the contrary a mouth-breather.

Schedule logic. Did I say schedule logic? I was once involved in a major project, where the putative project controls contractor had put together a “schedule” in OpenPlan®. Problem was, he had constrained every single activity, both at the beginning and end, and had no – and I do mean none at all – logic ties between activities. When the software refused to calculate the critical path (no doubt sensing the knuckle-dragging intellect behind the scheduler), I was called in to help analyze. When I pointed out that the problem was that all of the activities were constrained, and there was no schedule logic, the project controls contractor geniuses responded by … kicking me off of the project. They actually called in a friend of mine, Brian, from an East coast location of my company, and took him to lunch after having shown him the files.

“So, Brian, what do you think is the problem?”

“All of your activities are constrained. You need to remove the constraints, and add schedule logic.”

“That sounds interesting, Brian. Sell me on that.”

“I’m not going to sell you on that. That’s the way it’s done!”

These are three little things that make me crazy, and I’m sure y’all have more to share. Here’s your chance. Oh yeah, and buy my book (http://www.ashgate.com/default.aspx?page=1751&calctitle=1&pageSubject=956&title_id=11616&edition_id=15149).

That was great of my old friend, Stanly Raspberry, Private Eye, to convey a story in my last blog, about how useless it was to have risk analyst Myron Tittle tag along with him on his last case, but I’m afraid Stanly’s points were a bit subtle. So, I’ll be a little bit more blatant: much of what is presented as modern risk management is fraudulent, and a complete waste of money and time.

I offer a more thorough debunking of current risk management theory in (Marlin Perkins moment alert!) my new book, Game Theory in Management, Modeling Business Decisions and Their Consequences (http://www.ashgate.com/default.aspx?page=1751&calctitle=1&pageSubject=692&sort=title&pagecount=2&title_id=11616&edition_id=15149), but, for this blog, a couple of well-placed arguments should get the discussion rolling (or roiling).  I want to start by picking off that most irksome of risk analysts’ assertions, that “risk management” also includes “opportunity.” Note the gratuitous inclusion of the word in the Project Management Institute’s (PMI®’s) PMBOK Handbook Series on the subject, Project and Program RISK MANAGEMENT, A Guide to Managing Project Risks and Opportunities (PMI Publishing, 1992).  The risk management aficionados simply love to claim that their Gaussian curve-based models can somehow quantify how the future will unfold, so, naturally, they simply had to change the definition of “risk” to include the good with the bad. Hence, the coining of the term “upside risk,” and the inclusion of the management of upside risk, or opportunity, in subsequent PMI publications that deal with the subject. Here is a brief list of reference works that exclude any mention of “opportunity,” or, indeed, anything but hazardous or negative events being associated with the word “risk”:

·         Webster’s New International Dictionary, Second Edition, Unabridged (this book weighs 16 pounds)

·         Webster’s New Collegiate Dictionary, 8^th Edition

·         The American Heritage Dictionary, 2^nd College Edition

·         The Oxford English Dictionary (which, incidentally, included the word’s etymology as coming from the Latin phrase “to run into danger.”)

But, let the Project Management Institute issue “guidance”with an utterly re-defined “risk,” and all right-thinking managers must immediately adopt the new version!

Current risk analysis techniques rely to a high degree on the experiences of the project’s experts to provide best-case, worst-case, and most-probable scenarios to serve as the basis for the estimators’ cost baselines and contingency reserves. Take away the statistical analyses, the confidence intervals and Monte Carlo simulations, and this is what you have – not what went before, but what those creating the baselines perceive as what went before. Our internal narratives – including the ones that explain why the past unfolded the way it did – are full of cognitive biases, rendering the strength of our causality analyses far weaker than we believe them to be (hence, quasi-surreal events like the celebration of Groundhog Day). Why would anyone believe that, once we flip these narratives forward across the Time Now line, they can provide a workable structure for how the future will come about? And no amount of Gaussian-curve-based models overlaying these narratives can magically bestow validity to them.

As I discuss in Game Theory in Management, it’s another example of a management information system far outstripping its epistemological boundaries, and its adherents being placed in the position to having to promise to be able to return information concerning the unfolding of future events, which is obviously absurd.  Nassim Taleb, in his wonderful work The Black Swan, The Impact of the HIGHLY IMPROBABLE (Random House Trade Paperback Edition, 2010) had these two gems:

·         Also, many readers (say, those who work in forecasting or banking) do not often understand that the “actionable step” for them is to simply quit their profession and do something more ethical. (page 334)

·         This proves that everything relying on “standard deviations,” “variance,” “least square derivation,” etc. is bogus. (page 355)

And yet, a Google search on the term “risk management” returned over 63 million results (June 17, 2012), many of them organizations that offer to perform risk management services. I find it ironic that when I write about the accuracy and reliability of Earned Value information, I invariably receive comments along the lines of “It’s only a tool! It has little to do with project success!” But, let me cast any dispersions against the, in my opinion, vastly over-sold efficacy of risk management techniques, and the exact opposite happens, and to a significantly higher degree. In my previous gig, writing The Variance Threshold column for PMNetwork magazine, people would write the managing editor demanding my firing for daring to state that the definition of risk had nothing to do with “opportunity.”

So, that being my experience, I believe there to be a high probability of this blog generating some lively discussion – unless, of course, the risk management types, after having read this piece, are willing to concede the argument.

And I find that unlikely.