Project Management Metrics: The Hot Topic at #GartnerPPM

 

The Gartner PPM and IT Governance Summit was held in London last week. Lars Mieritz, a Gartner analyst, gave a good presentation about using metrics for communicating project success and effectiveness. He started out by saying that the top concerns of C-level executives and their direct reports, based on their research, were:

• Demonstrating PMO value
• Effective reporting and executive dashboards
• Measuring the business value of projects, programmes, and portfolios.

Gartner has many years of research in this area and he shared an interesting trend: in 2003 the top concern of CIOs with regards to IT strategy was the need to improve governance and provide leadership. In 2013 the top concern was delivering business solutions. There’s been a shift towards integrating IT more effectively with the rest of the business.

Why metrics count

Project management metrics based around portfolio performance help improve the perception of success. When researchers asked people what they thought of their PMO the views were pretty poor:

• 29% said they got some value out of the PMO but it was largely bureaucratic
• 33% said it was an integral part of getting things done
• 28% said it was a useful admin/support function.

As Lars said, he hasn’t spend decades in PPM only to be considered a useful admin function, so it’s clear that PMO customers believe there is certainly room for improvement in the services the PMO provides.

KPIs, said the verbatim comments that formed part of the feedback of this study, don’t link to business strategies. They don’t talk about shareholder value. Metrics relating to output don’t explain how to improve this output. Metrics don’t identify or clarify issues relating to performance. And finally, business management teams can’t relate to technical measures.

So, PMOs have plenty of metrics, but PMO customers don’t understand them or think they are useful. What should we be doing instead?

Creating good metrics

The main takeaway for me from this section of the presentation was that you shouldn’t copy someone else’s dashboard. What works for one business isn’t going to work for another, so templates aren’t actually that useful here. Every PMO is unique and provides a variety of bespoke functions to the rest of the organisation, so you should tailor the reporting and metrics in use to reflect that.

Metrics, Lars said, should reflect a strong customer focus to ensure they are well-understood and take a business view of project success. How is success viewed? he asked. Then shape management communications (i.e. reporting and metrics) along those lines. He offered some questions to ask yourself when putting together the relevant metrics for your PMO:

Is there organisational acceptance for standardisation around a single approach or method of reporting? This is particularly relevant if different business units have built their own measures and techniques over time or due to mergers and acquisitions. Different techniques mean different results.

Are the definitions of KPIs and metrics simple and useful for external comparisons? Even if you don’t need to compare performance of your PMO and projects with other companies right now you can be that someone will ask you to benchmark performance at some point in the future. Make your life easy and prepare for that now.

• Can you use data from other departments to save reinventing the wheel?
• Is all the underlying data readily available and more importantly, credible?
• Does everyone understand why you are gathering metrics and the purpose and objectives of this initiative?
• Are there sufficient resources to put together a full, comprehensive metrics plan? Do you have enough people or budget to support any training required?
• Is there senior management buy in? Setting up metrics is, after all, a project and it needs executive sponsorship.

Implementing project metrics

Once you’ve defined what metrics are relevant for your PMO, you then have to go ahead and implement them. Lars recommended engaging with users to ensure they understand the process and how these metrics will help improve things. For example:

• Average time lag between identifying an issue and corrective action – shows you delays in addressing problems and then you can work on speeding this up.
• % of projects with complete documentation – shows you where projects might be at risk due to information not being gathered; identifies project managers who many need additional support.
• % of projects with a status report over 10 days old – shows you where project data is out of date and who needs a kick to remind them to update it.
• % of total effort required to reach end of first phase – shows you potential for project to deliver on time and accuracy of planning/estimates.

“Good dashboards show where the problems are,” he said. And that gives you the chance to show what you are doing to eliminate the problems.

Dashboards should track what is important. He also had a great recommendation for PMO leaders who have to produce short reports for their colleagues or sponsors: if you don’t have much time to present statistics (or many pages to present on) then cycle round the metrics you show each time. For example, average number of training days per project manager won’t change much from month to month so show this one month and then report on something else next month. Over time it will give you the chance to report on more metrics, and of course if someone wants to see the training days figures they can always ask for them.

In summary, Lars concluded that metrics should enable us to take action, otherwise what’s the point of them? He said that we should seek metrics that are simple, intuitive and focus on goals and that finally metrics don’t replace judgement. You’ll always have to apply your contextual knowledge to a dashboard to interpret the full situation.

I attended the Summit as a guest of Genius Project.