The Relevancy Conundrum

 

No discussion of data disruption (ProjectManagement.com’s theme for July) would be complete without addressing that most difficult to define, yet central to all management information components, relevance. Dr. Watson was often surprised by Sherlock Holmes’ lack of awareness of popular news, which Holmes would explain as his desire to not overload his mind with data that was not relevant to his ability to solve crimes, demonstrating that the issue of data disruption has been with us since (fictional) Victorian London.

There are lots of definitions of the words relevant and relevancy, but, for the sake of this discussion, I’ll use the direct ”sufficiency to infer the conclusion.” What conclusion(s)? The one(s) that can provide the basis for, in our case, informed Project Management decisions. Recall my oft-stated derivative of the Pareto Principal, that the 80^th percentile best managers with access to only 20% of the information needed to obviate a given decision will be consistently out-performed by the 20^th percentile worst managers who have access to 80% of the information so needed. In other words, the existence and availability of relevant management information easily trumps superior managerial judgement, all other things being equal. It’s the reason why junior managers, generally speaking, do not have nearly the latitude of action that their more senior counterparts do – if one is not sure of the capability of a given decision-maker, limiting their options in an environment where canned strategies haven’t been tried is always the safer strategy.

Many things can chip away at an information stream’s relevancy. Perhaps the most pernicious is the insertion of subjectivity into the data set. Throughout my career, every single time I’ve been asked to perform an analysis of the circumstances and events that led up to a surprise overrun, the presence of subjective data – namely, the so-called “bottoms-up” Estimate at Completion – has been the culprit. This method of generating an EAC involves re-estimating the project’s (or Control Account’s, or Work Package’s) remaining work, and adding this figure to the cumulative actuals. The folly of employing this method is best laid bare with a simple histogram, showing three bars per reporting period, typically stretching back 6-7 months prior to the reporting cycle that (finally!) fesses up to the overrun: one bar indicates the Budget at Completion (BAC), the second shows the Project Manager’s / bottoms-up EAC, and the third shows the calculated EAC, based on the data available at each reporting cycle. Each time I’ve performed this analysis and produced said histogram, the results have been the same. The BAC level remains level (or close to level) for the duration, barring some Baseline Change Proposal that got snuck in to try and alleviate the incoming disaster. The PM’s / bottoms-up EAC mirrors the BAC, right up until two reporting cycles prior to the obvious, where it ticks up a bit, and then jumps to the same level as the Calculated EAC has been showing all along. The level of subjectivity inherent in the PM’s EAC, or the estimators’ take on the remaining work, render that version of the EAC irrelevant to the task of alerting management of an upcoming overrun. I call this effect pernicious because, not only does it fail to perform the task of identifying and quantifying cost performance issues early on, it actually does the opposite. It hides such problems, all while a valid calculated EAC is available, if management was only canny enough to reject the bottoms-up imposter.

Lack of timeliness will also erode an information stream’s relevance. “Quickness is the essence of the war,” claimed Sun Tzu,[i] and lack of it is poison to the Management Information System. The history of warfare is filled with stories of commanders not receiving critical, tide-of-battle-changing information until it was too late. Actionable PM information has a much shorter shelf-life than, say, information coming out of the general ledger, due to the more dynamic nature of Project Management over Asset Management. A Control Account Manager turning in a travel voucher late is no big deal. That same CAM who reports the percent complete figure late can disrupt the entire project’s reporting cycle.

Finally, if we are to employ the relevancy filter to eliminate otherwise disruptive data, then the comprehensive data set must be accurate. Inaccurate (i.e., false) data rarely comes with a warning label. But make no mistake – the processing methodology that can produce relevant information without accurate data has not been invented, nor will it ever be.

“Data! Data! Data!” he cried impatiently. “I can’t make bricks without clay.”

                                    -- Sherlock Holmes, The Adventure of the Copper Beeches

…and so it is with us PM-types. But the types of information streams we use are different by type, not degree, from those used by the Asset Managers, or even Strategic Managers, and the distinction will forever fall upon the filter of relevance.

[i] Retrieved from https://wealthygorilla.com/35-powerful-sun-tzu-quotes-art-war/ on July 11, 2022, 18:14 MDT.