We’ve all got metrics we use to assess project success: cycle time, earned value and so on.
As the year ends, maybe it’s time to look at some other measures we could use that might be a bit more… dare I say… interesting? Below, I’ve suggested 7 alternative metrics you could put in place (some easily, some would take more thought and set up) to look at what project performance really means in the round.
You might find CSAT in use across other teams. Why not implement it for project management customers too? Even if you work in-house, you will have internal customers. Trust me, they have an opinion on the project management service you provide. Why not check in with them directly and ask for it?
You don’t need a formal CSAT tool. Set some survey questions and set up a form to ask stakeholders their views, and then collate the results.
Cycle time is worth knowing, but does the end of your cycle always end in value delivered? A different way of thinking about it would be time to value: how quickly the project delivers tangible value per feature, or perhaps overall.
This metric comes with the added challenge of having to define value: but that could be a very useful exercise for stakeholders!
Could you create your own innovation index? There are already indices in use like the Global Innovation Index, but that’s probably overkill for our projects. Consider how innovative the product/deliverables are and the method used to implement them.
Here’s one you can probably get from your project management software but I don’t see it on reports very often. What could you take from a utilization report? Metrics are only helpful if there is something you can use them for, like decision support. In this case, it would be making sure the team is adequately resourced, so you really want to be looking forward not backward. Although historical data is useful too to see if there is a trend towards over or under staffing.
Could you create a metric that looks at how quickly the organization is adopting new changes? If you work with a change manager, they might have some ideas about how to implement this. Any new process changes or anything that requires training could be included, even if your measure was only based on smiley faces!
Your procurement team might already have a sustainability index based on their work with vendors and a sustainable supply chain. If you have an energy team, they might have measures you can pull into your projects too. For example, how much carbon saving your project is creating, or how much waste is recycled from different locations.
We mitigate risks, but are those actions really useful? We could draw on AI-powered insights by plugging in risk mitigation activities across a selection of risks and the outcomes. (Or you could work through this manually). I’m not sure how you’d assess the usefulness of the mitigation strategy: maybe on a scale of 1-5? Then you could see which actions had the biggest impact in reducing the risk.
There are lots of ways to measure project performance, and no one wants to be creating reports and tracking metrics for the sake of it. However, it might be worth looking at whether your current suite of metrics truly give you the complete, holistic picture of performance, because we all know it goes beyond time, cost and quality.
I have an electronic copy of the PMBOK® 7th edition, so from time to time I open it up to check on something. Recently, I’ve been looking at different ways to forecast as we’ve got some work on that needs to be planned out.
There are 6 quantitative forecasting options called out in the Guide. These are as follows.
Estimate to complete (ETC)
This is top of the list and the one I personally use the most often. It works even if you are not in a full, compliant, earned value management environment. The risk here is that we assume past performance is indicative of future performance, and honestly, why wouldn’t you? Unless you know something is definitely going to change measurable performance, you would assume that work is going to continue at broadly the same rate. Just jot that down as an assumption so it’s transparent to everyone.
Estimate at completion (EAC)
For me, this goes hand in hand with ETC. It’s calculated by taking the actuals and adding the ETC, so again, while it comes under the umbrella of earned value acronyms, it’s completely accessible to those who don’t work in EV setting.
Variance at completion (VAC)
As forecasting tools go, this gives interesting data. It’s the measure that shows the amount of forecasted budget over or under at the end of the project, and it’s one most project sponsors will be interested in: “Will we have any cash left to do anything else when we’re finished?”
To-complete performance index (TCPI)
I have never had the opportunity (or reason) to use this forecasting metric. Perfect for those of you working with earned value day in, day out, it’s the cost performance required to meet whatever management target you’ve set for the work. It’s a ratio, so I think it is less meaningful to execs who are used to see tangible numbers of days or money.
Now more and more tools are introducing AI features, it is possible to access regression analysis more easily. Perhaps you’ve got access to an AI-powered tool that will crunch these numbers for your automatically, removing the need for statistical knowledge.
The output allows you to predict performance going forward based on what has happened in the past, so it’s arguably more grounded than other guesstimates!
The final forecasting technique mentioned is throughput analysis. This looks at the number of items completed in a fixed time, so it’s useful for teams measuring features completed, velocity and story points. You can compare the output to those of other teams, although I’d be wary about comparing teams unless they work on very similar products or services. It wouldn’t be fair to judge a team on their throughput when dealing with very complex features against the performance of a team that has higher throughput but lower complexity.
However, the team can compare its performance against itself: that would be a worthwhile exercise. Ideally, you’d want to see that the learnings from retros have been fully incorporated and, more importantly, that the changes have actually made a difference.
Which of these are most used for your project forecasting? Let us know in the comments below!
You probably measure a range of things on your project. I’ve seen PMOs track number of open risks, projects closed this month and other numbers that are pretty much meaningless out of context. Here are 5 ways to make your metrics work better and give you more useful information, inspired by the Measuring What Matters report from PMI.
1. Measure more often
A study by PMI and PwC found that only 41% of PMOs were consistently measuring and reviewing performance. If you don’t measure regularly, how can you monitor trends?
They also found that around half of PMOs spent time communicating to the C-suite about milestones and project impacts. I know that in some organisations the PMO doesn’t have a direct line to the execs (although I think they should). Improving the perception of the PMO relies on the right people having the right information.
Action: Make sure you have a regular schedule for measurement so you can capture and track trends.
2. Collaborate up the organisation
Another thing highlighted by the PMO study was that metrics are often set by people who are not the PMO and who are not C-suite execs. Whether you are a project sponsor, senior leader or project manager, make sure you involve the right people in the conversation about what should be measured.
More collaboration between the PMO and delivery teams and the executives responsible for setting strategy should mean that you are measuring stuff that demonstrates whether the organisation is getting closer to the strategic goals.
Action: Check your measures are in line with the strategic vision for the company and that the right people were involved in coming up with them.
3. Focus on outcome-based measures
Outcome-based measures are those that reflect what was achieved on the project in terms of deliverables and change. They are different from the project management measures of time, cost and scope. (Note: those are still useful, but they aren’t the only thing you should be tracking.)
Projects exist to deliver change. That’s what is important: the impact that the work has on the organisation.
Action: Review the measures you are using to track your project and see which of them are outcome-based. Is that enough?
4. Review measures regularly – and with the right people
Once your measures are set up, keep them under review. Make sure to get input from a wide range of stakeholders: those who are collecting data for the measure, those who are using the measure for decision-making, the PMO and the project team, along with anyone else who has a stake in the outcome.
Reflect on whether the numbers or results are telling you what you thought they would. Do they provide accurate, reliable data that can be applied to different projects in a meaningful way? If not, why? Perhaps the measures need tweaking to make sure they properly serve their purpose.
Action: Schedule time to reflect on the usefulness of the measures you have in place.
5. Define success
Finally, define what success looks like. What parameters for your metrics represent a ‘good’ score? Below what level would you need management intervention? What are the red flags that would push the project into an Amber or Red status?
Capturing data is one thing – but being able to use it is something else entirely. In addition, some of the more ‘fluid’ desirable outcomes like cultural change or customer satisfaction are harder to grasp with a single number. You might need to combine several metrics to get a full picture of the current performance levels.
Action: Review the parameters that trigger action for your measures.
In this part of my occasional series on program cost management, inspired by The Standard for Program Management (Fourth Edition), I’m talking about 9 financial management tasks that you should expect to do as a program manager.
First up, be aware of what might be coming over the horizon to influence the budget baseline and what factors are leading to changes. This is basically being aware of what might create a change so you can be on the look out for any changes. It’s proactive listening and always considering what a course of action might do to your budget.
Part of the daily work of a program manager is keeping your eye out for any environmental factors that might impact the program overall, and budget changes are part of that. These could be environmental factors like changes in regulation, access to skilled resources from the local hiring pool, or access to the right technology – anything that could have an impact on the budget.
What happens after you scout the horizon for things that might change your program budget? You find things. Part of a program manager’s tasks is to manage those changes. The change could be to scope, to the schedule, to the resource profile, procurement plan, or anything else, and we have to play back what impact that is going to have on the budget and build in the changes.
As I mentioned in my last article, part of my role as a program manager is being able to shift funds between pots to ensure the overall program goals are met. That’s what this task is all about: monitoring the impact of what happens when costs are reallocated and making sure that overall the program components are balanced – or at least as balanced as they can be.
I mentioned cross-project impacts in the heading, but programs often include an element of BAU work and that can just as easily be affected by changes.
Some suppliers might be contracted at a program level, so it’s the program manager’s job to ensure that suppliers get paid in line with their contracts. With one supplier I worked with, we had contractual milestones in place. When each milestone was reached, another stage payment was due.
Part of my work was ensuring Finance was aware that a stage payment was due so they could have the right amount in the bank accounts ready to pay when the time came.
If your program uses earned value management, you will work with control account owners, project managers and other experts on the team to make sure EV practices are implemented and adhered to. Part of the role will also be making sure everyone can interpret the outputs of EV reports and manages their part of the schedule accordingly.
It’s unlikely that your program will be perfectly on budget every month because that would rely on your estimates being perfect and no ‘real life’ happening during the work. It’s more likely that you’ll be a bit under or a bit over.
Part of the program manager’s role is to manage within that tolerance and to advise and support project managers on the program to do the same.
Another big part of program management is communication and engagement with stakeholders around changes. The sponsor, customer, client and governance groups (including auditors) need to have access to transparent information around the program’s financial measures. You have to provide that.
Finally, the main, most obvious part of the role is to manage the program’s spending within the levels expected. You may have set tolerance at individual component level, or there may be other ways you are managing to within the expected parameters. These should be clear, so everyone knows how the work is being tracked.
In Part 1 of this article I talked about two ways to measure discrete effort: percent complete and fixed formula. Today I’m going to share the other two options: weighted milestone and physical measurement.
As a recap, discrete effort is the name given to the work required for an activity that can be planned, measured and ends up with something specific as the output. If you are doing work that directly leads to the completion of a deliverable, that’s discrete effort.
There are different ways to measure this type of work in a project using earned value management, or even in projects that don’t apply EV measures but still need to track progress (like: all of them). I wouldn’t use discrete effort measurement techniques for every single task on every single project, but they are available to you if it makes sense to use them. This is the benefit of tailoring 😊
The first one to look at today is weighted milestone. This is similar to fixed formula in that the value is apportioned to the work as the activity progresses, but it’s based on milestones.
Divide the work package into measurable chunks that are attached to and end with milestones. For example, a work package might include a couple of weeks of design work before a document is created. There could be a milestone at the end of the last of a series of design workshops.
When those milestones are reached, the activity earns the progress based on whatever breakdown you have assigned to each milestone.
So how is this different to fixed formula? It’s different in that it suits longer term work packages. Fixed formula is best for short activities, those that don’t stretch over more than two reporting periods. Weighted milestone progress tracking is an option for when your work package runs longer than that.
Ideally, each milestone should be a tangible ‘thing’ as well: some kind of interim result.
Reporting works best when there is at least one milestone within each reporting period, otherwise you aren’t able to tell if work is on track as it won’t have moved since the last report. You don’t get credit for any deliverables leading up to milestones where the work isn’t completely finished. In other words, it’s the achievement of the milestone that triggers the tick in the box, not the progress against the tasks leading up to it.
You don’t have to equally split the work across the different milestones. The ‘weighted’ part of this way of measuring progress allows you to apportion value across all the milestones. Allocate the appropriate amount per milestone, but don’t worry about limiting yourself to a regular split if that doesn't work for you.
Physical measurement is a way of tracking progress for things that can be counted. If you are working on something that has a specific, tangible way of measuring in a physical way, then this is the option for you.
If you can measure it specifically in a real world way, then that becomes your measure. You should agree it in advance so everyone knows how the effort spent will be tracked and reported on.