Project Management

Project Management Central

Please login or join to subscribe to this thread

Topics: Earned Value Management
How effective is EVM?
Do you use EVM (earned value management) for monitoring and controlling, or do you use something else? To what extent do you think EVM is effective?
Sort By:
Page: 1 2 next>
I don't use EVM. You'll probably get a different answer from someone who works in the public sector, but I've only found value in using EVM when working with third party agencies - consulting firms with contracted hours and deadlines. Even then, it was just for their work, not the whole project.

I still use some of the same steps - tracking actual and budgeted hours and costs can be important (I've worked at companies that don't care about this for internal employees) - but I don't bother with indices or s-curves because nobody wants to see them, they don't tell me what I don't already know, and, to be honest, EVM is only as accurate as your data. What percentage complete is a task, really, at any given point in time? How many hours is it going to take to find, fix, and retest bugs in code that your developers haven't encountered before?

I'm sure there are others who have worked through these concerns and find value in using EVM. I'll use it if and when I'm required to (which hasn't happened in 20 years of managing projects), or if I'm working with a third party agency. I'll be watching this post to see if others can provide convincing reasons/conditions for me to consider using it more regularly.
I use EVM but on its own, it is not enough as while it gives you indication of where you are in terms of budget and schedule, it doesn’t help determine if you are delivering value or not.

For example, you might be ahead of schedule but not delivering value to the client.
I find EVM is rarely effective, although I am required to use it frequently. I often refer to it as "doing" EV rather than "using" it.

EVM emerged from the US Department of Defense in the late 1960's, which is around the same time lots of systems theory and performance management concepts were born. That means it was originally intended for projects that were primarily physical in nature rather than software-centric.

The classic cost-curve over time describes how the ability to change a project outcome diminishes rapidly with time, as the decisions that drive most of the costs into a project happen early. This is logical, since physical systems require early design and purchasing choices well before components can be manufactured, and the integrated nature of systems means that the cost of change includes not only the intended change but often to many interfacing system components.

Because it is difficult to correct a project off-plan later in the lifecycle, EVM was intended to find problems within about the first 20%, although I don't remember if that's duration, or total cost. Early in the program lifecycle on physical systems however, there is little to measure. There is a lot of study, and experimentation, but few artifact releases and measuring the former is rarely effective. The problems show up later, at which time EVM is more about measuring how far off-plan a program has become, rather than an effective tool for correcting the course.

The other issue I find with EVM is teams tend to game the numbers when there is little to accurately measure. Teams will describe at length all the issues they are working through, but somehow magically, their SPI and CPI are 1.00000, clearly showing that they are just calculating where they should be in a spreadsheet, and putting that in the EVM system. Avoiding that is more of a cultural issue than one of logic, but people do tend to figure out how to get the attention off themselves to prevent unwanted "help".

I think EVM can be used effectively, and it is frequently considered a program best practice, but I rarely actually see it used effectively in practice. I find it tends to increase the total overhead on a project to manage the status reports, without providing actionable information.
If you are in the construction industry you can use, in addition to the EVM, the LPS (Last Planner System)
Yes, EVM can provide a global glance of the project situation. You can use S-Curve diagram to appreciate if the project is behind/ahead schedule or cost. But, as Rami says, it won't help you to determine if the project is delivering value.
Abdullah -

EVM is a good method of normalizing scope, schedule and cost to prevent a silo-focus on just one or two of these dimensions, but it can be gamed, it does have a number of prerequisites (e.g. an actuals reporting system which can report at the same level as your work packages) and it ignores things like benefit realization, stakeholder satisfaction and (to a certain degree) quality.

Kiron
I use EVM, however, the focus is on deliverables and their due dates as well as KPIs.
In my world, we are required to use EVM on our large government contracts. Like most PM tools, you get out of it what you put into it. It’s not a panacea, but it absolutely can give real benefit when used consistently—and honestly. Like someone said herein, you can game the system if you want… but then you have to ask why are you using it in the first place?

On my last big project, we ran the EVM numbers every single month, but then we met with the individual SME and group managers responsible for the various major WBS elements to understand what those numbers actually meant. We did this every single month for eleven straight years. Imo, it was very useful, but it took serious commitment and hard work.

For example, take how we dealt with variances. Above a certain threshold ($150K for us), any variance (plus or minus) had to have an accompanying variance explanation. For these variance explanations, we asked that the SMEs/group managers write the first draft of the explanation. (A good way to do this, btw, is to use a 4-sentence What-Why-Why-How template: What is the problem? Why is this problem occurring (i.e., root cause)? Why does it matter to us (i.e. impact)? And How are we going to address the problem (i.e., proposed solution). Solutions could range widely, from wait and see, to crash the schedule, to apply contingency, etc.)

Anyway, the SME was required to create the first draft of an explanation, and then would meet with our PM team, where we would review, understand, and then tweak/adjust the response as required. Then, the next month, we’d follow up to ensure that we were moving in the right corrective direction for that particular variance, and make further adjustments as required. In other words, we made EVM a tool for us, and not just something we “had to do” for our stakeholders.

Note however, that you can’t rely solely on EVM numbers to tell you completely where the project actually was. We met monthly with the SMEs and worked on the numbers problems, but I’m also a big advocate of “management by walking around,” so I usually personally traveled to and inspected the work, talked to others working in each area, and got my own “feel” for situations in addition to looking at the numbers. Said more simply, you must do both quantitative and qualitative assessments of the work to stay on top of a project.

The Bottom Line: Yes, EVM is effective. The method is a powerful and useful tool, but only if you treat it seriously, consistently, and use it in conjunction with other quantitative and qualitative tools in your PM tool box.
EVM is a very useful tool in the infrastructure project delivery industry BUT relies totally on being able to measure and document actual performance in real time. And then its only meaningful as a comparison against the expected performance as presented in the current budget and schedule. Typically EVM does not have a quality component nor reflect any future re-work.

EVM can be more of a risk than benefit if not rigorous in its application and interpretation.
​​I've seen a few replies herein that state EVM "does not have a quality component" and "ignores... (to a certain degree) quality." I truly don’t understand this. In fact, it feels to me like a misunderstanding of how EVM works. EVM, by definition, is a measure of a project’s progress. This means task completion, which means scope delivery, which in turn means quality being met.

If an element of your WBS is not complete and has not met all of its quality requirements (form, fit, function, performance, etc...) then you cannot take credit for it earning full value. Period. Instead, you must assess it at some amount less than 100% complete. Unless and until the item is fully delivered and meets all of its quality requirements–or is granted a formal waiver against un-met requirements–the earned value reported must be some objective value less than 100%. And once you have that percentage determined, the EVM calculations can be performed accurately. The trick to this of course is assessing an accurate percentage complete.

A properly set up EVM system at the start of a project should include the “rules” to be used for assessing the percentage of compliance. The formal name for these rules is the earned value “technique.” Where people struggle is determining a fair and accurate means of assessing percentage complete that removes human subjectivity from the equation. E.g., if a task is simple, like painting a fixed length of wall, then the value earned is simply a measure of the wall length painted– and accepted by the customer. But most tasks and scope of a project are a bit more complicated than this. Worse, the people assessing the completeness of the work (often engineers, SMEs, and work package managers) are human, and come to the table with their own levels of optimisms and bias.

That said, there are lots of ways to remove subjectivity and improve objectivity. For example, you could assign values with rules like the following: 0%=No Work Started On A Specific Task or WBS Element; 25%=Work Started (i.e., there is real value in the momentum of starting an activity); 50%=Engineer Reports That Work Is Done; 75%=Verification That Scope Created + All Acceptance Testing Performed and Requestion For Waivers Written/Pre-Approved; 100%=Customer Signs Off On Specific Scope Completion, Including Formally Approved Test Results + Waivers..

This is just one method. There a lot of ways to do these percentage assessments, but regardless of the method, quality absolutely has to be built into your EVM system as it’s integral to assessing the true completion of the work.

If you’re not factoring in quality, you’re doing EVM wrong.
...
2 replies by Keith Novak and Peter Rapin
Aug 03, 2022 10:49 AM
Peter Rapin
...
"E.g., if a task is simple, like painting a fixed length of wall, then the value earned is simply a measure of the wall length painted– and accepted by the customer."
Even a simple task can be more complicated than one may assume. In the example provided measuring the wall is only part of the 'completion' calculation. Are all the materials on site? Is access in place for the entire task? If materials and access are included in the costs and time (effort) than completion needs to include such. Will quality issues be revealed in the longer term? Typically the customer/client will not sign-off on partially completed work thus acceptance may not be available until well after the 'completed value' is determined.

You are right in that completed value is subjective - even when you assign values (0, 25, 50, 75, 90%). When dealing with many concurrent tasks the 'judgement factor' can be significant, especially when addressing quality.

One way to address quality issues is to set up a Quality Account where a percentage of the project estimated cost and time is assigned for re-work and quality resolution, say 5 to 10% of the initial budget - deals with peeling paint which manifests later in the program.

In order for EV to be a useful management tool it must be timely and, that being the case, not only does one have to determine progress but must do so quickly and repeatedly.

I am not arguing that EVM should not have a quality component, I'm just observing that it typically doesn't
Aug 03, 2022 12:23 PM
Keith Novak
...
Factoring in quality can be difficult in some industries.

In manufacturing, most of the engineering definition is complete well before there are components to inspect and test. Any quality issues from the design are already baked into the cake. That's part of why the cost of late change is so high. It requires new parts or rework, not just a paper change.

Since engineering and manufacturing are different skills, facilities, and timeframes of the project, EVM is used to measure engineering completion and fabrication/assembly uses different metrics. Discovering issues with how the fully assembled product performs isn't complete until the very end.

But then admittedly...I've typically seen EVM used to observe progress, but do little in the "management" part.
Page: 1 2 next>  

Please login or join to reply

Content ID:
ADVERTISEMENTS

"Man prefers to believe what he prefers to be true."

- Francis Bacon

ADVERTISEMENT

Sponsors