Last month I looked at a range of alternative metrics for assessing success. One of the comments on that article asked how, on reflection, had I seen these or other alternative metrics implemented effectively. It also asked whether there were challenges that might arise integrating alternative metrics into existing project management frameworks (which I’ll look at in a different article).
So, if you want to implement a more nuanced approach to measuring project success, how do you put this into practice? Below are some ideas. This topic is one that we could probably speak (or write) about at length, but hopefully the examples and ideas I share will give you some starting points for incorporating a range of measures into your project management practice.
This is an area where I have quite a lot of examples to share. In my book, Customer-Centric Project Management, my co-author and I describe an extensive exercise to set up customer satisfaction tracking on projects, with internal customers.
I’ve written about customer satisfaction measurement on this blog before (start with this article: Lessons about project metrics). Basically, find out what is important to people, then track that regularly and plot your scores as part of regular reviews.
You don’t have to use an ‘official’ CSAT mechanism, or pay for electronic survey tools. Just ask people as a minimum.
I haven’t used an innovation score regularly throughout a project, but it has been included in the way we rate projects and prioritize which ones should be done as the requests come into the pipeline.
Implement an innovation score yourself on a simple Likert scale (1-5 or similar). Ideally, you don’t want the measure to be subjective, so set some criteria e.g.:
Or similar – you really only need a couple of measures to make up your innovation score. Use the innovation score as part of the decision criteria for whether a project should be taken forward or not.
Resource utilization reports are something I’ve only actively used on a couple of projects, because mostly we either haven’t had the tools to extract the data (because we aren’t doing timesheets or detailed estimates), or because staff have been 100% assigned to the project so we don’t have to worry about them splitting their time across projects. We would still have to worry about them being scheduled to do too many tasks in a week and being over booked, or under utilized, but when the team is full time somehow that’s easier to manage as you can see what they are doing and we speak every day.
Use the reports to check exactly that: is the team going to struggle to meet its commitments? And if it is, because staff are over-scheduled, what are you going to do about it?
Even the basic reports from Microsoft Project will give you utilization data, so take a look at what you already have within your tool suite and if you find it useful, use it.
The final one I’m going to talk about today is tracking risk mitigation effectiveness. I mentioned in the article that you could use AI-powered insights to establish the effectiveness of the risk management activities undertaken, but in reality I imagine that takes quite a lot of effort to set up.
Another way to do this would be at project closure, or during lessons learned conversations, whenever these take place in your project lifecycle.
Look at what the risk was and what was done to mitigate it. Score the risk based on the effectiveness of the action:
Using residual risk (and specifically, a financial value of residual risk) is a way to establish how effective your management action was – or at least, it’s one way to create data that allows you to assess that.
Hopefully that gives you some pointers for measuring project success through different routes, with some more concrete examples of how to get started.
Project documents (and there are some good templates here on ProjectManagement.com) are important to keeping projects moving, and many times, a project will start with a business case.
You might accept the need to do a business case as part of the organisational process – just something you have to do to tick a box. Maybe your organisation doesn’t use them in a formal sense, but each project has to be justified in some way – whether that’s a slide deck or even an email. There is some ‘reason to work’ that kicks off a project.
But have you ever really stopped to think about what role a business case really plays? If you do them, I think we shouldn’t take them for granted. If you don’t do them, it’s time to start.
Here are a few reasons why it is advantageous to have a business case before the work begins.
Understand the scope
The process of putting together a business case helps everyone involved understand what the scope is going to be. And if they don’t like what that looks like, they have the opportunity to influence it early so the scope better aligns with the direction they want to take.
Understand the issues
Perhaps there are concerns, issues, risks or challenges that decision makers need to be aware of – there always are. The discussions that feed into the business case help make sure that everyone is aware of what those are and what implications they might have for the work.
Fact-based decision making will give the project a better chance of success. The leadership team can weed out the ideas that won’t work before any time and effort is spent on them.
We can frame this conversation by thinking about project viability. Having a thorough discussion of the issues makes people aware of whether a project is viable and will continue to be viable throughout the delivery phases, despite any challenges that may arise.
Finally, a good business case lays out information that is useful for managing the work, monitoring and controlling progress. For example, a schedule of stage payments or key milestones, scope elements or deliverables.
The business case isn’t the project schedule and you will need more than simply the business case, but if it is a well-thought through, well-prepared document, there will be enough in there to help set up adequate project tracking.
The document should also set out success criteria and/or benefits which give you the framework for evaluating success as the project progresses.
As a project manager, you might be thinking that putting together the business case is not really your job, and you’d be right. However, on the projects I have worked on, it’s always been easier to get up to speed and start work when I’ve been involved from the business case stage or earlier.
That doesn’t mean doing loads of work – just being interested and talked to and maybe asked an opinion about the resource information or timeline that should go into the document.
Then when I come to lead the work (assuming it gets approved), I have a better understanding of the ‘why’ behind the project and the decisions that have already been taken.
I’ve just finished reading Business Resilience by David Roberts etc al. It’s a book that sets out a whole framework for delivering progress at a sustained pace and not being left behind in VUCA times. It’s aimed at senior leadership at the top levels as that is where the culture change is likely to need to start from, if you are rethinking strategy delivery.
It did get me thinking about what it means for projects and project managers. Thinking about what resilience meant in an IT world, back in the days when I worked in the IT team, I could draw a few parallels with resilience from a tech perspective and what it translates to for project professionals. (These are my thoughts, they aren’t from the book, which is far more strategic and articulate!)
Process resilience to me means having steps in place to get things done, even when things change. For example, when I moved from a fixed term to a permanent contract, my records were updated. Unfortunately, that meant that I could no longer see any purchase orders that were approved by the ‘old’ me.
That wasn’t a huge problem but as process steps go, it would have been nice to have the continuity without having to ask for it.
Another example of building in process resilience is making sure workflows can be delegated when you are out of the business. For example, handing over order approvals, estimates or change management approvals to a colleague during your holidays, instead of them all being stuck in your queue to approve when you get back.
In IT, we used to build solutions that had adequate redundancy. For example, the servers would fail over to another server if the first server had problems. We had back up generators to keep critical systems operational if (when) the power went down.
In project terms, that would look like having two people trained to carry out a project role so that if one resource is off sick, someone else can step in and do the work.
That’s quite an overhead for a project team, as normally we wouldn’t want to carry additional cost, but on business critical projects, or where your resource is truly specialised, it might be worth it.
How long do you spend looking for project documentation? Probably quite a long time, especially if its on a collaboration tool that shall remain nameless! Thank goodness for being able to search.
In a technical environment, we’d create backups so the data was available even if the main system went down (although of course with the redundancy, the goal is that the main system stays up…).
Project documentation and data availability in a resilient team would mean you could find what you are looking for easily, in the right place, and access the data.
I think as the world gets more complex, projects get busier and teams have more to handle, being resilient is more and more important if we want to get things done and avoid burnout. These are just 3 ideas of things you can do with your project team this week to be a little bit more resilient and prepared for what next week might throw at you:
What do you think of these ideas? Share any other resilience-building tips you have in the comments below!
James Lea, founder of Project Science, spoke at EVA 26 earlier this year. He talked about the psychology of estimating. “People,” he said, “are just as important as the techniques and data.”
He went on: “Plans and estimates are built by and used by people. Psychology matters.”
The talk was very interesting, and here’s what I took from it.
He started by asking us our experiences of estimating and the emotional responses we had at the time. Think about your own experience of estimating. Did you feel:
That’s all (unfortunately) normal, and we all nodded along as he talked.
Challenge how will estimates be used
James talked about how we should challenge how estimates should be used. “Uncertainty drives variable reactions in our teams,” he said. “It drives emotions and responses.” If you are open about how estimates are going to be used and how they should be used, that can help people feel more comfortable with the process.
Make estimating positive
How can we enable our teams to experience planning and estimating as a positive, creative experience? Instead of the stressful, “I suppose I can give you a number,” experience that it is mostly?
It’s hard for an organisation to accept that it doesn’t know the answer, and that can sometimes lead to a poor experience of the estimating process for the people involved.
Here are some ways he suggested we could turn the experience into a positive one:
Creating a route to predict the future
James talked about asking the question about whether we have a route to predict whether the estimate is a robust one or not. We need to understand what is in and out of our control. Where things are out of our control, accept that and track it.
Estimates are only a guess without a map of how you got there and a set of viable routes.
We often hear that people can’t estimate where there is no historical data. Well, data science should make it easier now to estimate from past performance and the vast tracts of data we store about projects. If leaders can give teams the data, in a way that helps with estimating, that should make our estimates better.
Building defensible plans
James talked about showing your workings and documenting the bases of estimates. Steve Wake, the conference chair, shared his thoughts too, namely that the audit office regularly says people don’t know the basis of estimate and therefore the best ‘proof’ that your estimates are good is that you can justify them.
He talked about bounding your plans carefully, describing the world around the estimate as well as the estimate itself to provide rigour.
He suggested we quantify and compare with data science, applying risk appetite to the delivery methodology to round out what we know.
That, and the other points discussed, are ways to shape the emotional response and create a safe space for people to estimate their work.
What do you think? Let me know in the comments below.
1. Get involved with business cases and proposals
First, lobby to get involved with business cases and proposals. The more delivery expertise we have involved in the initial stages of a project, the more likely it is that the project will be able to actually hit its goals.
Have you ever been involved with a project where you’ve been handed something to do and the sales people have made promises that you can’t deliver on? Then you’ll know what I mean!
When project people are involved in business cases, in my experience you’re more likely to end up with a timescale that’s achievable and a resource plan that reflects the real amount of resources likely to be consumed by the work.
It’s even better if you can lobby for a seat at the table when the 3-year plans are being drawn up, so your top level project people, like the Head of PMO, get involved in creating the strategy in the first place. That provides a real insight into what initiatives are coming and how the delivery teams can help.
2. Use the project assurance function as a check mechanism
Project assurance is a way of checking that what you think you can do is actually achievable. It’s their job to pick apart your plans and proposals and apply a sense of real-world pragmatism. They can also help identify whether there are any resource gaps, strategic holes or other issues that you haven’t seen.
After all, as a project manager I’m sometimes so close to the information that I can’t see the bigger picture enough to realise that this project will clash with something that someone else is working on – but project assurance has the bigger picture and can point that out.
If you don’t have a project assurance function, ask a colleague for their opinion and talk them through the plans, asking them to basically pull them apart. Ideally, to be able to add some strategic oversight to your plans, this should happen around the time of the business case or PID. By the time you’ve got to creating a schedule you’ll be looking for a different kind of peer review.
3. Share what you know – but only what you know
My third tip is something I learned from Tony Meggs, Chief Executive of the Infrastructure and Projects Authority in quite an old article now that he wrote for Project magazine, but it has stuck with me. He said: Only announce what you know.
We know this in theory, so it’s not news to you, I’m sure. However, many project managers are ‘encouraged’ to share dates before we are ready, or pushed into committing to dates publicly before we have all the information to support the fact we can deliver to them.
So, if you want to be a strategic operator, only share what you know to be achievable. That goes for delivery methodologies as well. Meggs talked in the article about not committing to anything unless you know it to be true, including how the work would be delivered. If you are going to partner with someone and there’s a robust contract in place, by all means announce it. But don’t announce your intentions before they are fixed in stone because if it doesn’t happen you’re then having to walk back on the messaging and that can be damaging.
We can do this as project managers on a small scale, by giving our teams the space to come up with the best methods and timescales before we announce them to sponsors, but also on a strategic level, by ensuring there is a communications plan in place that supports the bigger picture messaging for the project, programme or even the portfolio.
Do you do any of these already? How are they working out for you? Let us know in the comments!