As we approach the peak of the COVID-19 pandemic, part and parcel of the post-event analysis will include attempts to answer the question: how much did this cost us, and how much time did it remove from our productivity? Whether by nation, macro-organization, corporation, small business, or from the point of view of specific industries or families, the upcoming weeks and months will, no doubt, see many attempts at quantifying these impacts, usually in order to ridicule political opponents, but more importantly to ascertain those business strategies that are most effective at making the analyses’ target group more economically resistant to similar future occurrences. Unfortunately, both purposes of analysis invite quackery, and in massive proportions. Those who are supposedly looking to advance management science can be counted on to advance their own particular pet theories (although establishing how risk management [no initial caps] can help insulate organizations against pandemic shutdowns would be quite the reach, I fully expect something to be pushed that way). And yet, this serious question needs a serious treatment, regardless of charlatans muddying the waters. How can this be done, precisely? As fate would have it, I have some experience in this type of scenario, and the good news is that the solution is far simpler and easier to access than almost all of the alternatives often pursued, and that optimal approach comes from basic PM.
First, let’s dispense with the intuitive but nevertheless incorrect notion that estimators can deliver the information being sought here. They can’t, as is easily observed by considering the common practice among projects to derive a “bottoms-up” estimate at completion, or EAC. Back when I was working to attain my estimator’s certification I was taught that there were three types of estimates:
- Order or Magnitude, or “ballpark” estimate, derived by comparing your intended project to similar work done previously, and accounting for differences in size, function, geographic and inflation. The bases of comparison are published by organizations whose data has been shown to be consistently reliable; however, these estimates’ accuracy is rated between 45% and 60% of final costs.
- A Budget Estimate lists the intended resources by type (Heavy Equipment, Labor [by type], Overhead, Hotel Load, etc.), based on the most recent information for each category. Perhaps the most common type of estimate, these are rated at between 30% and 45% accuracy.
- A Detailed Estimate is usually produced by a professional estimator, using off-the-shelf software for that purpose, and is so detailed that it can be handed off to the procurement specialist to begin buying the labor and materials needed to work the project. This type of estimate is accurate to within 15% and 30%.
Keep that last accuracy bracket in mind as we proceed with this analysis – a professional estimator, using OTS estimating software, can be expected to be up to thirty points off of the real answer, and even the optimal scenario is 15% accuracy.
However, what’s being estimated here isn’t a project. It’s the impact of a Black Swan event (as described in the excellent book by the same name by Nassim Taleb) across a broad spectrum of organizations and projects. There is simply no reliable way of knowing what a “final impact” of any given occurrence, much less one with the massive, far-reaching impact of a pandemic, could have on any given subject. The trucking industry is being hard-pressed by the unavailability of the facilities that drivers need, and that can perhaps be quantified. But what about the impact of lower fuel prices? Yes, hotels are largely slowing down or shutting, but demand from on-line retailers is jumping. How to cross-evaluate these factors, and dozens, if not hundreds more like them? It’s simply impossible to do. Facilities such as restaurants can compare their pre-virus numbers to post-virus versions, but since when does any business, let alone an entire industry, experience level revenue figures across weeks, let alone months?
“Alright, Michael” I can hear GTIM Nation say, “how does one calculate the impact of a shutdown, as your title implies?” Here’s how it’s done, at least in project space.
Watch your Cost Performance Index (CPI) and Schedule Performance Index (SPI), at the cumulative level. For those new to these information streams, these are derived so:
CPI = BCWPcum / ACWPcum
SPI = BCWPcum / BCWScum
where BCWPcum is the cumulative Earned Value, ACWPcum is the cumulative actual costs, and BCWScum is the cumulative time-phased budget. What you will see happen is a sub-1.00 spike in both indices (1.00 is performance exactly as planned; below that number indicates trouble, for both indices) downward in the next couple of months, most likely more pronounced in SPI than CPI. Then, over the following few reporting cycles, you should see these figures climb their way back to where they were before, the speed with which they do so determined by the robustness of the subject organization. How accurate is this method? Studies have shown it’s good to within ten points. This method of impact quantification is not only accurate, but it’s best simply because it’s not predicated on the thousands upon thousands of parameters which, even if they were identifiable, cannot be reliably captured.
If you have no Earned Value Management System in place, two things: are you really doing Project Management at all? Secondly, well, I can’t help you.
Also, stay safe out there.