We’ve all got metrics we use to assess project success: cycle time, earned value and so on.
As the year ends, maybe it’s time to look at some other measures we could use that might be a bit more… dare I say… interesting? Below, I’ve suggested 7 alternative metrics you could put in place (some easily, some would take more thought and set up) to look at what project performance really means in the round.
You might find CSAT in use across other teams. Why not implement it for project management customers too? Even if you work in-house, you will have internal customers. Trust me, they have an opinion on the project management service you provide. Why not check in with them directly and ask for it?
You don’t need a formal CSAT tool. Set some survey questions and set up a form to ask stakeholders their views, and then collate the results.
Cycle time is worth knowing, but does the end of your cycle always end in value delivered? A different way of thinking about it would be time to value: how quickly the project delivers tangible value per feature, or perhaps overall.
This metric comes with the added challenge of having to define value: but that could be a very useful exercise for stakeholders!
Could you create your own innovation index? There are already indices in use like the Global Innovation Index, but that’s probably overkill for our projects. Consider how innovative the product/deliverables are and the method used to implement them.
Here’s one you can probably get from your project management software but I don’t see it on reports very often. What could you take from a utilization report? Metrics are only helpful if there is something you can use them for, like decision support. In this case, it would be making sure the team is adequately resourced, so you really want to be looking forward not backward. Although historical data is useful too to see if there is a trend towards over or under staffing.
Could you create a metric that looks at how quickly the organization is adopting new changes? If you work with a change manager, they might have some ideas about how to implement this. Any new process changes or anything that requires training could be included, even if your measure was only based on smiley faces!
Your procurement team might already have a sustainability index based on their work with vendors and a sustainable supply chain. If you have an energy team, they might have measures you can pull into your projects too. For example, how much carbon saving your project is creating, or how much waste is recycled from different locations.
We mitigate risks, but are those actions really useful? We could draw on AI-powered insights by plugging in risk mitigation activities across a selection of risks and the outcomes. (Or you could work through this manually). I’m not sure how you’d assess the usefulness of the mitigation strategy: maybe on a scale of 1-5? Then you could see which actions had the biggest impact in reducing the risk.
There are lots of ways to measure project performance, and no one wants to be creating reports and tracking metrics for the sake of it. However, it might be worth looking at whether your current suite of metrics truly give you the complete, holistic picture of performance, because we all know it goes beyond time, cost and quality.
Discrete effort is the name given to the work required for an activity that can be planned, measured and ends up with something specific as the output. The effort involved directly links to the delivery of the thing you are creating.
There are four ways of measuring progress on a task that is managed with discrete effort. Even if this description of what discrete effort is doesn’t make much sense to you, you’ll start to realise what tasks are appropriate to be planned in this way when you see how you track them.
The four methods of measurement are:
Now you see how to measure the work on these tasks, can you think of some activities that use discrete effort on your schedule?
I have to confess to spending many years using percent complete before I ditched it: it’s not really suitable for business change projects where the output doesn’t easily break down on a percentage basis. On one project we used weighted milestones, as that aligned with the contract agreement for billing.
All the methods have their place.
In this article I’m going to look at the first two: percent complete and fixed formula, and then in the future post I’ll dig into the others.
Percent complete is exactly as the name suggests: the measure is an estimate of how much has been done in percentage terms, tracked at the end of each reporting period.
Ideally, it should be based on some kind of measurable thing instead of just a number that the team member has come up with. Have you ever been in a meeting where a task is reported at 90% complete for several weeks? I have. In the end, Chris and his 90% complete became a running joke. There was always something his team was working on that was 90% complete – but not quite ready to be signed off.
With percent complete, the planned value (PV) represents the time-phased resource needed to finish the work package. Earned value is then calculated by multiplying the percent complete by the budget at completion for the work package. In other words, you simply use the relevant percent complete for the budget. If the budget for a work package is £100, and the task is 60% complete, the EV is £63.
That makes it easy to work out.
The challenge with using percent complete is that it is not easy to work out. You need to do so in an objective way, and that kind of goes against the grain for many stakeholders, especially team members who haven’t worked in a disciplined EV way before. They might be used to providing very subjective guesses for percent complete, and that’s really not what you are going for here.
Often people use hours worked as a guide for percent complete, but again that’s not always 100% accurate. You could have worked half the time but the deliverable only be 20% complete. The remaining 80% will be achieved with the remaining work hours. So you do have to be a bit careful about how percent complete is implemented – this is why we document how performance will be measured so there is no ambiguity.
The fixed formula method of progress tracking relies on there being a formula you can use (the clue is in the name). You assign a specific percentage of the budget value of the work package when the work begins. Then the rest of the budget (or time) is assigned when the work is completed.
A task starts. You assign it as 25% complete, in terms of budget and/or schedule. Then when the work is finished, the work package “earns” the remaining 75%.
You can do this as a 50/50 split or any other breakdown that works for you. Obviously, the total assigned to each milestone in the work package must equal 100%. You don’t have to limit yourself to an allocation at the beginning and then another at the end: if it makes sense to split the task into 5 and assign 20% of the value at each of those fixed points, then do that instead.
This is a good method for allocating value in environments using earned value management, or where you have to report progress at work package level but don’t have the data to track things hour by hour to give you an exact percent complete score. It’s also a very easy method to use. Once you have your formula set up and your assignments clear, you can just get on with doing the work. The performance measurement will be pretty seamless, as long as you are confident work is progressing to plan. And there’s no cajooling the team into coming up with measures that are a few percentage point higher than last week just to prove something has been done in the last 5 days.
Good for: short duration tasks.
Remember that the percent complete assigned isn’t reflective of the actual work done or costs incured. Stakeholders need to understand the limitations of this method.
There is a variation of the fixed formula method which is 0/100 percent: in other words, the task doesn’t ‘earn’ anything until it’s done. There is no progress or performance measure assigned to it. The work is either fully complete or not done at all.
This is a good option to use for deliveries or where the deliverable is coming from a third party and being tracked outside of your project. For those tasks, the activity is either not finished, or done. For deliveries, for example, the materials are either on site or they aren’t. There isn’t much point in assigning progress when they are en route, as that doesn’t really get you any closer to the end goal.
Fixed formula is a flexible way to think about performance measurement in earned value settings, but it’s also helpful for projects that are not using EV.
Do you use percent complete or fixed formula? Or something else? In the next half of this article, I’ll talk about the other two methods: weighted milestones and physical measurement. See you then!
Pin for later reading
Archiving Project Data
Categories: project data
Over the last couple of months I’ve looked at some of the tech trends affecting us as project delivery professionals working in an online world.
One of the technical challenges I haven’t talked about yet of using collaboration tools is how you archive the data effectively. Archiving tools are available, but they are yet another system to integrate within your technology landscape. There’s nothing to say that their development will keep up with the constant evolution of the SaaS marketplace; in fact, I think it’s fair to say that they aren’t.
Forrester reports that only 15% of businesses actively capture and archive data from collaboration sites (Hayes & McKinnon, 2015). The old approaches to data management and records compliance just don’t cut it with new communication channels, even where interoperability makes it possible.
This problem is going to get worse before it gets better. Regulatory bodies will catch up with the increase in data being stored across collaboration tools and online and will demand that companies manage their archives more effectively.
Organizations will be forced to adopt more robust methods of managing archives with the associated cost of data management that comes with this. Archiving strategies need to be built in conjunction with the adoption of online tools and with flexibility in mind.
All the trends I’ve talked about – like AI, robotic processing automation and interoperability - bode well for both the manufacturers of project management software, and (one would hope) the people using them. Better data, better collaboration, and better end-to-end systems should increase the likelihood of success on projects because it all contributes to better decision making.
And I believe collaboration tools are already improving the results of teams where they are being used – I certainly see that in the conversations I have with my mentoring clients and the project managers I work with every day.
However, the great results we expect from tech will only come about if the tools being applied meet a genuine business need. You can’t layer tools into a broken, uncommunicative team and expect them to suddenly work together, just like that. The team culture has to be ready and open to change, the infrastructure has to be there, the management desire to want virtual and online working to succeed has to be there – and people have to see the benefit.
The business need that drives all of that is likely to overlap the areas of technology, collaboration, and culture. The way people work online—both in and outside the project environment—is not perfect, and we can expect to see more evolutions and innovations in the years to come, both in terms of tools and the way in which interactivity is encouraged and fostered.
I hope that we will eventually look back and realize that this was the time that organizations made the shift to the collaborative project environment. While the societal change may feel fast, in organizational terms, it is infiltrating slowly. Project leaders are essential in supporting innovation and effective collaboration in all its forms.
This article includes a few points that were made in my PMI book: Collaboration Tools for Project Managers. Given what we’ve been going through and seeing so far this year, it felt appropriate to try to pick out some comments on tech for teams and where that might be taking us – because it seems to me that virtual working is here to stay.