Measuring the impact of continuous improvement
| Over the last few weeks I’ve looked at why continuous improvement matters in project management, how we create the right culture for it and what tools we’ve got available to do it (lots, fortunately). Now, I want to look at how you can check to see if those improvement opportunities are having the effect you expected. Again, we have project management tools and techniques readily available for this because we have methods for benefits realisation and tracking. And those are the kind of things you can draw on for tracking. But first, we have to start off with some metrics.
What metrics are you going to use?All benefits tracking relies on having something to track against, so when you are planning your improvement initiatives, it’s important to think about what you are going to use to track. How will you know if the change has been successful? In reality, some of the changes are going to be small and you wouldn’t track them. For example, I removed a section of a template that was no longer relevant – I’m not going to set up tracking to establish whether or not that was worth it. It was worth it. If in the future we decide we want it back in (we won’t), we can add it back in. But there are other changes where it is worth measuring the impact, especially if you have to justify the change in some way. For example, if you need investment to make the change (like a change to your software tool or to pull chargeable resource off a chargeable project to do an internal change) then you’ll want to track. You could look for: Process efficiency metrics: Track changes in key process metrics, such as time to completion, resource usage, and cost savings. Ideally, you would want to see a reduction in time or cost per task. So you have to have ways to track that today, and not be implementing anything else at the same time that would also affect the metric. Quality improvements: Measure the improvement in project deliverables, such as defect rates or rework levels. You’re looking for a decrease in errors or bugs. For example, if you’ve implemented improvements to software testing or pair programming. Team engagement: If you have employee satisfaction surveys you could use those results. Or you could look at team retention rates, or the number of improvement suggestions submitted by team members as an indicator of engagement with the continuous improvement process. However, a lot of factors influence engagement, so it’s hard to say with confidence that it’s the changes that have been made that have had a significant (or any) impact on engagement and morale. Also, the impact diminishes over time. While people might start out thinking the change is amazing, over time it just becomes ‘the way we do things around here’ and people start to take it for granted. Assessing the impactYou’ll need to find a way to track performance before and after. Alternatively, consider benchmarking against other organisations. I’d only suggest doing this if you already have access to the information. When we’ve run a benchmarking exercise, it’s been expensive and I’m not sure it gave us the level of detail that would help with project management continuous improvement. So I personally wouldn’t be recommending that we invest in benchmarking unless it was for something bigger, like an assessment of PMO maturity against industry competitors. Anyway, I digress. Qualitative measuresIt’s also OK to go for qualitative measures like stakeholder feedback. Ask if they think your improvements have made a difference. However, it also might be positive if they haven’t noticed a difference. That means your change has gone in so smoothly that it is transparent to stakeholders, and that might be a good thing! Be sure to ask the team as well. Continuous feedback loops are useful, and basically that just means talking to people about how the change has impacted them and listening to their feedback. I’m enjoying writing about continuous improvement, and next time I’m going to cover some of the challenges that you might face while trying to make project management process improvements, with perhaps a few tips on how to overcome those. See you then! |
Do you have your head in the sand about these project challenges?
| As much as we’d love to have everything working to plan, sometimes we can take some aspects of project management for granted. I wonder how many of these project challenges you have but you haven’t openly recognised within the team? I don’t want you to have your head in the sand, so below I share 10 things that you might be missing on your project. (I didn’t want to bury my actual head in the sand, so you got a photo of my feet buried instead! Not quite the same, I know.)
We might believe that stakeholders are reading our project comms, but are they really? It’s hard to tell, but if you aren’t seeing any action, or people are asking questions you have already answered in your comms, then they probably aren’t.
I expect it isn’t, even if it feels like it should be. People won’t turn up or won’t use the online training to teach themselves. Plan to have post-launch training support because your users will need it.
Hopefully they are, but as above, there will always be someone who says they didn’t get the memo, or couldn’t access the materials. Make sure you’ve got post-launch training in the diary as well, and do more change management activity and engagement than you think you should have to.
UAT is the one thing that gets squeezed in my experience. Add in an extra cycle if you can. It’s easy to take it out – not so easy to squash it back in if you do need more testing time. This obviously depends on your project – if you are doing something you’ve done a lot before you should have a high degree of confidence in your deliverables, but you might need more if you are working on something new.
It hasn’t! Make sure risks are constantly mentioned in all meetings, and that you are always listening out for what might be a new risk.
Team morale is something to keep an eye on. The team can get demoralised for the smallest of reasons, from a change not being approved to a reschedule for whatever reason. That negativity can spiral. Even if things are going well on your project, events outside the project can influence morale, such as a change in leadership, redundancies, an increase in BAU work or pretty much anything else.
We’d like to think the financial and tangible benefits are understood, but how well have those assumptions been documented? I’ve worked on a couple of projects where we think we understand what we are tracking, only to find that we can’t replicate the exact formula the finance person (who has now left) used, or some other difficulty. You should be able to answer straightforward benefits-related questions like ‘is this incremental revenue or total revenue’? So make sure you can.
There probably is, but people are too polite to mention it. You should dig for it! Some conflict is healthy so don’t worry about bringing it up.
Are you finding bugs in your deliverables? And more importantly, are you squashing bugs? What are your criteria for exiting testing and how does quality play into that? Ideally, you should have great quality measures, and be performing in line with those, but sometimes project teams get swept up in the desire to deliver and that means some lower-risk bugs are left in for now.
Are your project deliverables introducing technical debt? Are you breaking things elsewhere or for other teams, or introducing workarounds or degrading the solution for someone else? Sometimes your part of the world might be fine, but a process you’ve implemented ends up in many additional steps or a new report being needed for someone else. Check that you aren’t creating technical debt by accident – if you know about it, document it and have created it ‘on purpose’ as part of a stepping stone to a different solution, then that’s probably fine. Which of these might your project be at risk from? Let me know in the comments! |
