Before You Test The Software, Check This

 

I once worked with a large program office that had several, far-flung subordinate project offices. To keep up with the satellite office’s schedule performance, the home office had a software package that enabled the sites to send in updates pertaining to their attainment of key milestones. This program wasn’t based on Critical Path Methodology – instead, this “status” was in the form of assigning a category to the milestone: (1) completed, (2) expected on-time or early, (3) anticipated delay, or (4) anticipated major delay or out-and-out miss. Of course, this setup wasn’t a schedule performance measurement system at all, despite the way it was advertised. As my regular readers will readily recognize, this system’s design was essentially that of a poll: the performing organizations weren’t sending in performance data. They were, rather, transmitting their opinions on how they would perform, which is a very different animal. 

An entirely predictable pattern emerged: at the fiscal year switchover, a new set of milestones (with their due dates) would be negotiated, and all of the milestones in the report would show an anticipated on-time delivery (2). Then, as these milestones’ due dates drew within a few months, their status would tend to change to anticipated delay (3), and, only within one or two months of the due date, the activities that were genuinely in trouble would reveal an anticipated major delay, or a complete miss (4). A legitimate schedule performance measurement system could have alerted the home office of the problems in time for them to help deal with them; instead, the satellite offices were in a position to obscure any issue that might make them look bad in comparison to the other sites, and the home office remained in the dark until their intervention couldn’t help anything. But, hey, the software package itself performed as expected!

It was pretty obvious to me what the problem with this system was, and so I, along with two associates from my organization, visited the site that had developed the software in the first place. There’s actually an old scheduler’s trick for assessing performance in environs where routinely collecting the data needed to keep a critical path method package going isn’t an option. All you need to do is to divide an activity’s cumulative duration by its estimated percent complete, and you have a fairly accurate estimate of its total duration. Compare this figure to the activity’s original duration, and you know within about ten points if the activity will finish early, on-time, or (gasp!) late, and by how much. Since this system already had the activities’ start dates, all it needed was the percent complete estimate (instead of the milestone categorization used), and it instantly became an effective (if basic) schedule performance analysis tool. The programmers we met with thanked us for the insight, but turned down cold our entreaties to update their software. The reason? “Because before this package was introduced, there was nothing, and, even if it is flawed, this is better than nothing.”

So, what we had was an instance of a piece of software that had been adequately tested and installed, but was still failing at its purported function, that of monitoring schedule performance across multiple project sites within a program. Why was it failing, despite passing its tests? Because it was predicated on a flawed business model. Polls are not performance measurement systems, period, even though they often masquerade as them. However, once in-place, it would prove to be very difficult (if not impossible) to replace it with any system that would represent a superior solution based on, essentially, the flawed sunk-cost argument. 

So, before you test the software – indeed, before you code the software – do yourself and your customers a favor, and test the validity of the underlying business model. How can you do this? Well, first you…

Oops! Look at that, I’m out of space. If you can’t wait, check out the book that this blog is named after, available at http://www.ashgate.com/isbn/9781409442424.