Game Theory in Management

First, A Little History

I think the proliferation of capability maturity models (CMMs) owe their popularity to Carnegie Melon University’s Software Engineering Institute (SEI), which performed a study on how software companies mature in their ability to successfully produce computer programs. The analysis became part of the book Managing the Software Process by Watts Humphrey[i], published in 1993, that divided software engineering firms into five levels:

• Level 1 (“Initial”), where everybody is basically doing their own thing (or nothing) with respect to the sought-after capability,
• Level 2 (“Repeatable”), which is very basic but standardized, so the entire organization is performing at the same level of expertise,
• Level 3 (“Defined”), is the Level where, if the heroes who got the organization to this point are hit by the proverbial beer truck, the organization’s capability level doesn’t decline because there are sufficient procedures and training in-place to perpetuate that level of expertise,
• Level 4 (“Managed”) organizations are good enough to export their expertise to others organizations, and
• Level 5 (“Optimized”) organizations are so good at what they do that they regularly discover solutions to long-standing problems.

Have We Seen This Before?

I was struck on how similar these “Levels” were to Tuckman’s stages of group development[ii], published in 1965, specifically that project teams go through a four-stage progression, of Forming – Storming – Norming – and Performing. If the SEI model assumes the teams have already formed – they did, after all, examine existing organizations – then Level 1 equals Storming, Levels 2 and 3 represent Norming, and Levels 4 and 5 are marked by their superior performance. If we’re talking about assessing organizations by the performance of their personnel resources by categories or Levels, then the two structures, in my opinion, are remarkably similar. SEI simply adds more detail. Don’t misunderstand – I’m not accusing anybody of plagiarism, or idea-lifting. It’s just that the practice of defining categories of organizations by certain criterion (“There are two kinds of people – those who divide people in to two categories, and those that don’t”) and then using those criterion as a measuring standard isn’t new.

If this is the case, then consider some of the implications. Currently, most college-level business schools teach that the point of all management is to “maximize shareholder wealth.” Agree or disagree with me that this is clearly silly, there can be no valid argument that this approach is against the Project Management raison d’etre, which is to manage work in such a way as to satisfy the customers’ expectations/parameters of scope, cost, and schedule. But that never stopped the asset management crowd from trying to tell the PM aficionados how to do their jobs, based on the analysis they derive from data in the general ledger. There are actually several books on how to calculate the return on investment (ROI), a favorite analysis technique of the general ledger geeks, on setting up a Project Management Office. For generations the asset managers’ tools have been intruding into project management space, where they most certainly do not belong.

Take That, Finance and Accounting!

But along come the Organizational Project Management Maturity Model. And what does it do? It essentially points out that corporations that have many projects feeding into several programs becoming part of a portfolio of work can have optimal (and, by extension, sub-optimal) organizational structures. The step-by-step ideological payback follows this path:

• Organizations are made up of resources.
• Resource management is not Project Management.
• PM, however, provides the metric by which organizations succeed or fail in portfolio management space.
• Basically, we PM types have turned the tables on the Asset Managers, by redefining (correctly) how organizations achieve success (or are viewed as failures), and on our terms.

This form of epistemological payback was a while in coming, but it’s said that revenge is a dish best served cold.

Lagniappe

My next book is coming out later this month. It’s entitled The Unavoidable Hierarchy; Who’s Who In Your Organization, And Why, and it’s available here. It’s about how people tend to fall into specific roles, or ranks, in the organizations in which they participate, and an analysis of the tactics used to advance within those hierarchies. My former PMNetwork columnist colleagues Neal Whitten and Bud Baker both gave great reviews of the manuscript, leading me to believe it might be worth its price (at least on Kindle!).

[i] Capability Maturity Model. (2016, August 28). In Wikipedia, The Free Encyclopedia. Retrieved 16:55, September 5, 2016, from https://en.wikipedia.org/w/index.php?title=Capability_Maturity_Model&oldid=736597555

[ii] Tuckman's stages of group development. (2016, July 22). In Wikipedia, The Free Encyclopedia. Retrieved 17:08, September 5, 2016, from https://en.wikipedia.org/w/index.php?title=Tuckman%27s_stages_of_group_development&oldid=731055786

There was a funny German comedy show that featured a skit where a woman and her elderly father are in the kitchen when she asks him “So, papa, how did you like the iPad we got you?” The dialogue is in German, but it’s clear that he’s been using the iPad as a cutting board, as he uses a knife to scrape sliced vegetables into the container in front of his daughter off of the iPad, and then proceeds to rinse it off and put it into the dishwasher. The look on the daughter’s face is priceless, as she reacts to a comically bad misuse of the device.

Speaking of comically bad misuses of devices, on the Wikipedia page for risk management[i], there’s a graphic of the International Space Station, with the areas most at risk from debris impact highlighted in red, and text underneath stating “Example of risk management.”  Directly beneath that is the topic classification, “Business administration.”  I have to admit, it’s a brilliant ploy, aligning an engineering function with a business administration one in the risk management aficionados’ desperate attempts at glomming on to legitimacy.

But it’s another example of comically misusing a tool. Here’s why.

To an engineer, “risk management” has nothing to do with management. It deals with the characteristics of materials, which are knowable. Specific designs using specific materials under various conditions can be analyzed, and their performance accurately predicted. Selecting steel over aluminum in building, say, those areas of the International Space Station most likely to encounter debris impact is the result of quantifiable, repeatable analysis, based on the likelihood of debris striking that part of the station, the added costs of lifting heavier components into orbit, the ability of certain designs and materials to absorb impact, etc., etc.

Not so managing a project. Project teams are composed of people, whose characteristics most certainly cannot be precisely quantified, much less predicted under specific circumstances. As I pointed out in last week’s blog, even in iterations of the Ultimatum Game, where choices available to the participants was limited to one decision each, the analysts completely missed the most likely strategies employed by the players, instead predicting the tactic that almost never worked. With that being the case, what are the chances (get it, risk managers?) that, with the number of available decisions open to members of the project team, customers, and other stakeholders, the likely outcomes can be captured through statistical analysis?

Oh, risk management isn’t about calculating likely outcomes? Then what is it about, exactly? That’s one of the most frustrating things about debating this topic with their true believers. They start with this whole business about how risk management is about “anything that impacts the project, positive or negative,” but then fall suddenly silent when asked to produce any report based on their analysis techniques that actually helps inform project managers’ decision-making. “The odds are X that something bad will happen to your project” does not help your typical PM, and modifying that to “the odds are X.N% that occurrence Z will happen to your project” isn’t any better.

What’s clear to me is that the risk managers took a concept that was perfectly legitimate in the engineering realm, and converted it to the project management arena, where it most certainly does not apply. When challenged on this, the risk managers I know point to RM’s legitimate uses, which are not in Project Management. But – again – people are neither structures nor materials! Need more evidence? Consider that crucible of PM, the Agile/Scrum Project Team. One of the many blessings that Agile/Scrum brought to the management science table is that they burned away some of the trappings of traditional PM that tended to be so overdone, they bordered on being superfluous (e.g., highly formal change control techniques). So, you Agile/Scrum Project Team members – do you even have a risk registry? No? Could it be because such teams are based on the decisions and choices made by the team members themselves? They behave in ways that cannot be predicted, nor quantified, and those behaviors are the key determinants of project success.  The sleight-of-hand involved in conflating engineering uses of RM and its alleged role in project management, as intellectually vacuous as it is, has taken in so many in the business realm that risk management is a multi-billion dollar industry world-wide.

But it’s completely invalid. And, if you believe to the contrary, my only advice is to closely inspect any cutting boards your children gift you.

[i] Risk management. (2016, August 24). In Wikipedia, The Free Encyclopedia. Retrieved 01:23, August 28, 2016, from https://en.wikipedia.org/w/index.php?title=Risk_management&oldid=736035884

Since it appears that I may be the only risk management contrarian here at ProjectManagement.com, I can’t let my readers down when it comes to challenging and refuting the fevered rantings (strikethrough) proffered hypotheses of the other side. The last two postings have been pretty strong, in my humble opinion. This one will be stronger still, with my posting for August 29^th being the ultimate risk management overturn. For the penultimate RM refutation, let’s turn to game theory.

Game theory is predicated on the idea that many real-world circumstances and situations are analogous to more controlled situations and circumstances, where the behavior of participants can be observed, quantified, and evaluated, with the superior strategies or tactics becoming discernible. Simply transfer those strategies and tactics to their real-world counterparts, and you’ve increased the odds of success, right?

Well, not always. As I discussed in my second book, (after which, incidentally, this blog is named) a favorite “game” of theorists is the Ultimatum Game. Here, two strangers, Player A and Player B, are approached by a person with an amount of money (typically $100 USD), who informs them that the money will be simply given to both players, but if and only if Player A, on the first attempt, proposes a split of the money that is acceptable to Player B. If Player A proposes a split that is unacceptable to Player B, then neither player receives anything.

The pre-game predictions of the theorists were fascinating. Their conventional wisdom held that, in order for Player A to maximize their payoff, the optimal strategy would be to propose that As receive $99, with Player B receiving just $1, on the grounds that, given the rules of the game, Player B would rather receive an unearned $1 than nothing at all, which would be the outcome should Player B reject Player A’s proposal. But something very interesting happened on the way to Player A depositing their $99 – in real-life iterations of the Ultimatum Game, Player B almost never accepted that division. There were, in fact, many observed instances of people rejecting 50-50 splits, or even 40-60.

In the investigation as to why the predicted outcome so dramatically deviated from the actual ones, several factors came in to play that hadn’t been anticipated.  For those that rejected the $99 to $1 split, common responses included a sense of resentment at being taken advantage of by Player A, or perhaps Player B simply didn’t believe that Player A was ninety-nine times more deserving of unearned money than Player B. Some individuals came from cultures that abhorred even the sense of being in debt to other people, and therefore rejected the 40-60 division in order to avoid being perceived as being in debt to Player A. The reasons for Players B to reject the division were myriad, but one thing was fascinatingly consistent: the 99-to-1 division, predicted to be the winning strategy, was the approach most often rejected. Faced with such a dramatic refutation of their calculated, “best” strategy, many game theorists simply attributed it to “cultural” factors which, obviously, could not be quantified or evaluated in deriving a superior strategy in the Ultimatum Game.

The parallels here with current risk management theory are striking. As I have often claimed in this blog (and in columns, keynotes, books…) there are simply too many factors involved in pursuing project goals to be able to capture, much less quantify, the data needed to calculate the odds and impact of future project-impacting events, much less calculate an optimal strategy for dealing with them. Compared to the number of decisions that customers, stakeholders, and the project team can and do make, the choices presented to the players of the Ultimatum Game are profoundly limited, to one choice each, and even here those ended up being two choices too many to quantify and calculate into a winning strategy. But, just as the game theorists could punt to incalculable “cultural factors,” so, too, can risk managers point to the existence of “unknown unknowns” when their techniques fail to deliver anything resembling a usable information stream. Convenient, no?

Keep in mind, then, if you are ever asked to participate in a rendering of the Ultimatum Game, don’t do as the so-called experts recommend, and propose the 99-to-1 split – it won’t work. Similarly, if you are playing the Project Management game, and your risk manager recommends a strategy based on his statistical analysis, don’t do it, because… well, you know.

One of the definitions of faith is “firm belief in something for which there is no proof.”[i] It’s a well-known fact that many people, including project managers, believe that modern risk management techniques are a valuable addition to the information streams available to them. But is there any proof of this?

I wasn’t always a risk management skeptic, you know. In fact, at one time I was such a true believer that I wrote some software that pulled project data from a popular critical-path methodology software and performed a single-tiered decision-tree analysis of a project’s WBS (at the reporting level) in order to calculate the contingency budget. Its core formula looked like this:

Cb = ( ∑ (TA1B * OOTA1) + (TA2B * OOTA2) + (TAnB * OOTAIn) ) - PMB

Where Cb is the contingency budget amount, PMB is the Performance Measurement Baseline, TA1B is task alternative #1’s budget, OO is the associated odds of occurrence, TA2B is task alternative #2’s budget, etc. The task alternative’s budgets were estimates of the impacts of various things that could happen to that particular activity. Take their sigma, subtract out the original cost baseline, and you have a risk—based estimate of the amount that should be set aside for contingency.

Of course, to make it usable we had to auto-insert some assumptions. Since 68% of all data points fall within one standard deviation of the mean, we assigned those odds to the tasks’ original estimate. The most likely alternative outcome was given the nominal odds of occurrence of 27%, since that was the amount covered under the next standard deviation.  The fairly unlikely scenarios (worst case scenarios) got the next 4.7%, and the extreme outliers were – again, as preliminary place holders – assigned the last 0.3%. When interviewing the Control Account Managers to collect their insights as to the nature of alternative task outcomes and their impacts, we would always point out that the odds initially assigned were just boilerplate, and that they should feel free to alter them, based on their expertise and experience. Interestingly enough, they rarely did.

Something else really interesting happened during these data collection interviews. When asked about alternative outcomes to the planned tasks, the CAMs were clearly engaging their imaginations, and could admittedly only speculate as to the cost or duration of the alternative endings coming about. For those risk management fans who believe that Monte Carlo analysis provides a more robust (or even valid!) approach, something very similar happens here, too. Once the “most likely,” “best case,” and “worst case” scenarios are established, almost all Monte Carlo software packages for project management invite the analyst to select a distribution curve – usually a leaning bell – to serve as the bounding parameters for the “randomly-generated” alternatives’ cost and schedule impact. But then the exact same type of processing proceeds, just with however many “randomly-generated” additional data points.

To state the blindingly obvious, this is not management science. It is an invitation to speculation, tripped out in probability-and-statistics jargon. These speculations are not, repeat not based on hypothesis, testing, and validation or refutation. Instead, they are based on the projections of the various subject matter experts, or control account managers – in other words, they are largely faith-based, predicated on the idea that their current scope is sufficiently analogous to their previous work to generate an informed guess. Note also that most risk managers will insist that the risk analysis should continue throughout the projects’ life-cycle. Since alternatives that could impact the project are consistently popping up, this is not surprising; but it does point to a specific lack of finality that’s absent from the projects’ baselines. Scope, Cost, and Schedule, at some point, are all “frozen,” or, to one degree or another, contractually binding. Not risk, no siree. It’s almost as if contemporary risk analysis is more about dealing with an unquantifiable future than it is about processing actual data into usable information, the latter being an instantly recognizable aspect of management science, the former more in line with, oh, I don’t know, faith?

As long as I’ve gone this far, I may as well state the obvious conclusion: any system of beliefs that is largely faith-based is NOT management science, and far more closely resembles a religion. I’ve been hearing the faith-based message of the risk managers for decades now, and I’m more than a little sick of it. I’m ready for this service to end.

[i] Merriam-Webster on-line, http://www.merriam-webster.com/dictionary/faith, retrieved August 13, 2016, 14:18 MDT.

Hatfield’s Uncontestable Management Theory #9 is that any notion that can’t be clearly articulated, both as to what it is as well as what it is not, cannot be taken seriously, or even legitimately evaluated. An old business axiom (not one of mine) is that which cannot be measured cannot be managed; similarly, inchoate ideas aren’t even weak hypotheses, much less usable management science.

Which brings us back to our friends, the risk managers, and their ongoing assault on the English language. Last week I took them to task over this whole risk-equals-opportunity nonsense, but that at least had the characteristics of simply trying to manipulate the language in order to support an otherwise silly notion. What I wish to address next represents a full frontal assault.

A principal aspect of risk analysis theory (strikethrough) suspect assumptions is that future events can be categorized as either “known unknowns” or “unknown unknowns.”  I can’t help but to wonder what the person(s) who came up with this nomenclature was thinking, since these categories are anything but intuitively obvious. The future is unknown, and unknowable by mere mortals. Ah, but the whole risk management facade comes crashing down in a heckuva hurry if this fact becomes widely accepted. They, therefore, come up with these two categories, which, like the risk/opportunity conflation, are self-evidently contrary, attempts to bully those who know the plain meaning of the terms notwithstanding.

Which brings us back to the marginal concept of “unknown unknowns.” What are these? It’s not overly cynical to say that these are events that the risk analysts did not anticipate, and therefore couldn’t estimate odds of occurrence nor cost/schedule impact. Pretty nifty, no?, to be able to completely mis-judge a management situation, and pass off the failure by categorizing it as an “unknown unknown.”

Then we have the issue of the repetitive-contrary-term phrase. The prefix “un,” in the English language, means “not.” By definition, a thing cannot be both “N” and “not N,” any more than something can be both “known” and “unknown.” The very fact that risk managers would attempt such linguistic legerdemain is itself, stark evidence for the poorly-thought-out nature of their ideas. As for this phrase’s cousin, it’s simply repetitive. The future is unknown. Saying that a specific future occurrence is “unknown unknown” is sophomoric repetition; it’s unknown – that’s all that needs to be said about it.

Imagine a project to paint a house blue. As the PM sends his assistant to purchase the selected paint, he finishes erecting the scaffolding and is approached by his risk manager.

“I want to talk to you about your project’s scope, and risk registry” he begins.

“Umm, okay, whaddaya got?”

“Based on previous projects’ performance, I think you should prepare for the chance that you won’t paint this house blue.”

“Why not?”

“Because your previous projects were to paint houses different colors, meaning that we should prepare for the outcome here to be either blue un-blue, or un-blue un-blue.”

“What’s ‘blue un-blue’?”

“More like a green.”

“That’s not blue.”

“It is if you mix it with yellow. That’s why I called it ‘blue un-blue.’”

“But my assistant has the exact color formula.”

“Then there are the un-blue un-blues, such as red.”

“We’re not going to paint the house red.”

“But you have to prepare for the chance of that happening!”

“It won’t ‘happen.’ If my assistant comes back with the wrong color – any shade of ‘un-blue,’ – I will send him back to get the right color.”

“But we should plan for the cost or delays associated with that happening.”

“No, we shouldn’t. How much am I paying you, again?”

And our PM didn’t even ask about the odds of painting the house un-blue un-blue.