Don KimPROJECT-TO-PORTFOLIO MANAGEMENT EXPERT| Seeking opportunitiesSacramento, CA, United States
All,
I'm wondering if anyone has had practical experience utilizing earned value and statistical process control. Though Six Sigma is the latest implementation fad for SPC, I'm avoiding that moniker since I don't want to be constrained by it and am looking to elevate my PMO's project control methods using rigorous quantitative control methods.
I'm specifically looking to use it on one of my projects, which is to build a new IT infrastructure for an R&D site, which is comparable to another previous project I was engaged with which was to build a new IT infrastructure for our manufacturing site in East Asia.
I think EVM works best for large, globally dispersed, multi-million dollar construction and infrastructure projects to get a handle on costs and schedule. And for this project, I'm on the heals of just completing an equivalent project in sequence to apply EVM and SPC to see if actual performance and process improvements can be quantitatively measured.
I am aware though, of the flaws of EVM with regard to schedule performance metrics and am researching Lipke's Earned Schedule technique as well and possible integration.
Interested in your thoughts.
Saving Changes...
Sort By:
Wai Mun KooPMO Director| Intergraph PP&MSingapore, Singapore
Don, regarding the flaw you mentioned about EVM, I do not really think that it is a flaw though. We have to know that in EVM, we are always referring to the dollar and not time. SPI in this case is a measure of work done (earned value) versus planned work (planned value). Assuming we have a planned value of 400K to reach in the month of May, and we only managed to complete 300K (EV) in May, then we can only say that we have a schedule variance of 100K in May. We can't say or tell the variance or delay in time as we can't really predict exactly how long we need to complete the rest of the 100K of work (we can only estimate). In this aspect, I would say this is the way EVM interprets value (or I may say lack of visibility in time domain) and not exactly a flaw. Saving Changes...
Wai Mun KooPMO Director| Intergraph PP&MSingapore, Singapore
Don, going back to your question on quantitative approach in project, there are various statistical methods/tools in 6 Sigma that you may look at. As a matter of fact, most of the methods/tools in 6 Sigma are standard statistical methods/tools. You may choose not to follow the DMAIC or DFSS process and just utilize the methods/tools for your project needs. Saving Changes...
Don KimPROJECT-TO-PORTFOLIO MANAGEMENT EXPERT| Seeking opportunitiesSacramento, CA, United States
Wai-Mun,
Thanks for your excellent replies. On Six-Sigma, you are correct and in fact I'm using this to obtain my green belt in Lean Six-Sigma with my company. I just wanted to avoid titling this post as EVM and Six-Sigma, since that methodology comes with a host of statistical methods and tools that are not specifically relevant to what I'm trying to accomplish. Mainly, earned value through statistical process control. My main tools will be EVM, control charts and histograms.
One item that is missing from traditional EVM is setting a rigorous mathematical threshold for measuring CPI and SPI variance. EVM is ideal to use with control charts since the data is inherently "normalized" since the idea value always equals 1. The problems with population sampling and variability of data are largely avoided.
While I agree with you that "value" in EVA is largely in terms of cost, or what you earned based on what cost you planned to spend on your project, I think you must account for time variance as you will need to know from the statistics, what corrective action are needed to get you back within your confidence limits for time as well as cost.
Furthermore, these account for cost and time and are missing the third part of the triple constraints, namely scope. If we buy into EVM as being a more rigorous way to track project progress, then the normal Gantt chart and cost tracking method, then it must tell you in quantitative terms, if you are within the full scope of your project deliverables or if you did not plan well enough.
For example, since the ideal conditions for CPI and SPI are 1, then it stands to reason that your project if it is to fulfill all the triple constraints of time, cost and scope, then the ideal project will follow the S-curve with AC, EV, and PV following very closely and converging at the end of the project with all three measures at 1 with the BAC.
I think using SPC will allow you to know what thresholds of variability of time and cost your project can safely be in and to let you better forecast the probability of your project converging on 1 to BAC.
Now, how to rigorously map all this to scope in a quantitative manner is something I still need to look into, unless others know or have tried. I would love to hear that feedback! Saving Changes...
MITCHELL DAVIDPerformance and Quality Manager| WORTHINGTON INDUSTRIESWorthington, Oh, United States
Don,
Scope can be quantified; the precision of the measurement (and the desired or necessary precision) is what comes into question.
At the first level of quantification, scope = "things" delivered. If you plan to deliver 10 "things", then the project is not successful until you have delivered 10 "things"; quite straightforward.
But that seems too obvious and simple right? Exactly. That is why when people say "scope", they are often mixing "things" and "quality" in the same; the "things" are obvious, while the "quality" of those things is not (or at least it can be debated).
Quality (in a project) can be quantified as the degree to which the deliverable meets its specification. Have specifications, and test / measure all deliverables against those specifications.
But, what do you do when your deliverables exceed (or do not meet) specifications?
A really good project will have targets for schedule, targets for costs, and targets for quality; specifying a target for schedule and a target for costs are easy tasks, so that is what most project managers focus on. Specifying targets for quality (scope) is difficult, so most project managers do not focus there (or do not know how); it is not easy, but it IS possible.
Use the "Cost of Quality" model to target and quantify scope/quality. It really is more straightforward than it sounds. "Cost of Quality" equals the sum of prevention costs, appraisal costs, internal failure costs, and external failure costs. All projects absolutely WILL have a positive "Cost of Quality" (COQ) (if you spend zero on prevention and appraisal, you WILL have failure costs (for example).). You cannot eliminate COQ, but you can minimize it, and set targets for it.
Not all tasks on a project will be a COQ related task, but you do (or at least should) have tasks / activities planned for each area.
1. Plan to have prevention activities (actions designed to prevent spending failure costs). (Training, for example, is actually a prevention activity).
2. Plan to have appraisal activities (Testing, for example, is an appraisal activity).
3. Plan for internal failure activities (Internal failure activities are those you do to rework substandard deliverables BEFORE you deliver them to your customer.)
4. Plan for external failure activities (External failure activities are those you do to rework substandard deliverables that are delivered to your customer.
Specify targets for your COQ at various points in your project, track to these costs, and track the variance. At a particular point in your schedule, look at these costs:
A. If COQ is HIGHER than expected and Failure costs are HIGHER than expected, then you are NOT meeting your PLANNED scope (cause you are reworking a lot of stuff?). If your failure costs are higher than prevention and appraisal costs, you need to spend MORE on prevention and appraisal to reduce your Failure costs, and your overall COQ should decrease, and you deliver to scope.
B. If COQ is HIGHER than expected and Failure costs are LOWER than expected, then you are probably delivering to your PLANNED scope, but could reduce your prevention and/or appraisal activities; but do so carefully as to not spike your Failure costs, and increase your overall COQ.
C. If COQ is LOWER than expected then you are probably delivering to your PLANNED scope.
Track your variance to "cost of quality".
[I capitalized "PLANNED" in the response, because I want to connote that the level of quality on your project is really neither the right nor the wrong level; it is really more about delivering to the level of quality that your customer is willing to pay for, and that you have successfully negotiated with your customer.].
See the American Society for Quality website for more details. (www.asq.org) Saving Changes...
Wai Mun KooPMO Director| Intergraph PP&MSingapore, Singapore
Mitchell, fully agree with you. When I started to use EVM for my software development project years back, i did realize that the quality control part is missing. Therefore, I introduced another index QPI - Quality Performance Index, to go along with CPI and SPI. Approach is very similar to what you have described. However, I measure the cost of quality based on the cost of rework that I need to get the deliverables back to scope and meet the stakeholders' expectation.
Don, I am not quite clear when you mentioned 'scope' in your reply. Mitchell stated well that scope is equivalent to the work to be delivered in the project which could be represented by WBS and mapped to cost as BAC. The granularity of the scope description affects the differences between the expected deliverables vs. the actual deliverables, in other words the quality. Would you kinldy explain a little bit more on what do you mean by 'how to rigorously map all this to scope in a quantitative manner'? Saving Changes...
Don KimPROJECT-TO-PORTFOLIO MANAGEMENT EXPERT| Seeking opportunitiesSacramento, CA, United States
Mitchell and Wai-Mun,
You both bring up an excellent points and perhaps the quality index you both advocate is what I am driving at. Like most project circumstances, I don't think the quality metric or my proposed scope metric will work to quantify all projects, but the one I'm applying it to I think is perfectly suited for EVM and SPC.
For my project, I have to deliver a physical infrastructure, IT equipment and systems and relocate and migrate all users from an existing site to the new one at very determined times and with a very clear expected outcome. Therefore, for example, if I am tasked with having 8 network switches installed in 1 month with each week having 2 at a time installed, turned on, tested and ready to go, then the measure of EVM in this circumstance can be well quantified.
Just for the sake or argument, let say in this instance that I have met my scope and was able to deliver each set of switches within cost and schedule performance. But would this apply to quality? If the quality plan required that the switches output and input a certain bandwidth of data and the test revealed they don't, then my quality index would not have earned what it was suppose to, but I did meet project deliverable scope?
Anyway, this may be getting to the point of splitting hairs on these metrics, but this is a very interesting topic and I'm grateful to have such sharp PMs provide their inputs.
This is the first time I will apply EVM with SPC and I can see I have a lot to learn.
Thanks for the feedback. Saving Changes...
Don KimPROJECT-TO-PORTFOLIO MANAGEMENT EXPERT| Seeking opportunitiesSacramento, CA, United States
I just realized that it was over a year ago when I posted this question, and I'm now finally underway having executed the project and am using EVM in conjunction with SPC to monitor and control this project. I'm using the full gamut of EVM and control charts from SPC to gauge the variations in CPI and SPI indexes. This is all being tracked in Excel with as much automation I could program in.
Some observations on using these techniques:
- In order to have accurate and rigorous data, I've been tasked with really pushing the teams to provide me with an accurate WBS and to schedule out when those work packages will be delivered
- Due to the economic landscape, this project was re-scoped and "value engineered" down which mean lots of scope changes and re-baselining of the project schedule which made the process more difficult.
- Using these techniques are not for the faint or heart or those PMs not willing to drill down in the details. I've really had to stay on top of checking the cost and schedule progress against the WBS and scope/requirements baselines, more than I can ever remember on my past projects!
- Though techniques like control charts are really used to measure variances in high output manufacturing, applying this to my EVM measurements provides me another rigorous quantitative heuristic on how well my project is in control and is mapping quite well with EVM.
All in all, I'd have to say it has been worth it as my project is ahead of schedule and well within budget. Interestingly, I have the WBS in a line item form in Excel and I added a column to track the items left to do against the time they were to be done and when this was output graphically, it became a kind of WBS burn down chart. This is much like the kind used in Scrum to track the backlog items that have been completed.
When I compared this to the EVM s-curve graph, you could see the correspondence of items completed with the costs associated with them. I could see the nice inverse relationship between the downward burn down WBS chart and the upward EVM s-curve graph.
Topic to be revisited! :) Thanks Wai Mun and Mitchell for your feedback and insights, though I've yet to consider how to incorporate the QPI index as I've got my hands full as it is! Saving Changes...
Wai Mun KooPMO Director| Intergraph PP&MSingapore, Singapore
Don,
Congrats that you have a successful project. Just one question. How do you attribute the success of your project to help of using EVM when you said 'my project is ahead of schedule and well within budget'? In other words, how EVM has helped you to achieve that? Saving Changes...
Don KimPROJECT-TO-PORTFOLIO MANAGEMENT EXPERT| Seeking opportunitiesSacramento, CA, United States
Hi Wai Mun,
As I mentioned in my post, implementing EVM required having very detailed and planned out inputs (WBS, plan, schedule, baseline, etc.) so that the outputs (EV analysis and SPC control charts) would have accurate analysis and data. This process in turn, allowed us to come up with a solid cost and schedule baseline with the appropriate contingency reserves, schedule buffers and anticipated team resource (as well as vendor and consultant) needs so that we could keep the project within schedule and budget.
There were other factors of course, such as having good executive support and sponsorship (since its a big budget project) and the fact that we did a very analogous project a couple years ago gave us a historical baseline to work with.
In this instance, doing EVM made perfect sense because we had the most ideal situation to make this process as defined as possible (well, I'm also going to get a green belt at project completion so I was motivated to drive this initiative :)).
I'm still about 4 months away from the go-live date, but the EV analysis is forecasting "green" for the triple constraints and the variations are well within the upper and lower limits defined in the control chart tracking SPI and CPI (I'm using classical Shewhart charts and tracking within 3 standard deviations of errors).
