Project Management

Project Management Central

Please login or join to subscribe to this thread

Topics: PMO, Quality
Have you been involved in the creation of a Quality Framework to measure Projects within a PMO?
Hello everyone, besides the common triangle variables (cost, time and quality) which other factors or KPIs will you include in a Quality framework for projects in a PMO
Sort By:
Arturo -

1. Realized business value

2. Stakeholder satisfaction

Dear Arturo
Interesting your question
Thanks for sharing

Customer satisfaction

many are possible, only a few are useful, depending on your specific situation.

You should first determine your quality objectives for projects though and the stakeholders you want to satisfy. There is no use to count something if nobody is interested.

Another thought is are the measurements identical for all projects (e.g. agile/rollouts/infrastructure), or do you need several sets of KPIs?

Then balance the measurement costs vs there relevance for the different stakeholder groups (e.g. a team might not care about ROI but the portfolio manager will). If you ask each project manager to maintain or report a lot of measurements each week or month, you will soon get pushback and the PMO is doomed.

Possible KPIs I have seen
- ROI from business case
- Team Morale index
- Sponsor satisfaction
- compliance with project company standards (must criteria)
- meeting attendance rate of key stakeholders
- PM satisfaction with PMO
- test quality
- function points, project size
- CPI from earned value, EAC
- velocity in Scrum projects
- complexity level (a combination of e.g. #of team members, business areas, interfaces, changes per month/dynamics, ..)

There is a deck on slideshare 'Building the PMO as a Program' that describes a stakeholder / benefits oriented approach to PMO deliverables. Ask if you want more support.

I build my first PMO in 1995.
The key success factor is KISS: keep it simple.
Check out this link, as there are some really good ones IMHO.

you could also add the following:

1. Number of technical/ engineering changes requests (in scope and out of original scope by PM and by Contractor). Include any associated delays and costs.
2. Number of contract change requests (by PM and by Contractor). Include any associated delays and costs.
3. Delays in project deliverables (by PM and by Contractor).
4. Lessons Learned that were actually implemented from previous projects. What were the savings in time, cost, scope, and quality. Looking forward, projects should as part of their benefits realization plan actively pursue previous lessons learned, and include specific ones that they will implement as a project goal(s)/ objective(s)
The first PMO's I was involved in focussed on, naturally enough, the competency and consistency of the Project Manager and PMO proactices. All good stuff at the time - PMO produced the standards and templates and measured the PM's use and adherence of them for quality.
Over time I have noted a trend away from this approach because it tended to perpetuate problems rather than resolve them. Project focus has shifted, and is in the midst of changing to be more responsive to the business demands. Therefore the measures have to change, and a good PMO should be responsive to these changes. However many PMO's I have seen have failed because of rigid adherence to outdated (for the business) standards and measures. PMO's should constantly be asking the business, and building into project requirements the measures that will, in a real-world sense, determine what success means once the project is finished. Measures should exist not only during the project life-cycle, but at the end of the project and at 1 or more cycles afterwards. If business did not get value from the project then there should be an effort by the PMO to define with the business what the measure should have been and how it can be improved in future. PMO's should never fall back on static definitions - that way lies extinction.
I apologize, in advance, for a long response. Hopefully it makes sense and is helpful.

A quick online search will produce dozens of KPIs for measuring PMO and project performance. I avoid using the triple constraint for KPIs, because:

...they reflect stakeholder interest and guide decision-making: is the sponsor more interested in cost or schedule?
...they are variables based on when they are establised: can you accurately identify how much a project will cost, or how long it will take, before requirements are complete, if it's something that hasn't been done before?

I'm not saying the triple contraint shouldn't be measured. They are among a number of metrics that provide meaningful information that can help you make decisions, even if they don't tell you how effective your team is at achieving business objectives (which is what "they" say KPIs are supposed to tell you). Consider the following:

- How may times the schedule changed. What number of schedule changes should be allowed?
- How many times the budget changed. What number of budget changes should be allowed?
- # of critical defects/blockers. How many is too many?
- ROI. How soon will ROI be realized? Is it incremental, or lump sum? Six months, or two years? Will the PMO/PM have any involvewment in realizing ROI, over time?

These measures can tell you that something did, or didn't, happen. But, they don't tell you why. Further research may reveal that initial estimates were made too early, so the resulting changes were just expected refinements of estimates as the project progressed. Or, they may highlight poor 1) decision-making, 2) project selection, and 3) market knowledge on the part of those who approved the project (which nobody is likely to admit to). This could be helpful, and might help lead to changes in expectations and demands, but doesn't really tell you much about the performance of the project.

Some measures can help explain the "Why," but they are often secondary measures. Being able to demonstrate how many hours your personnel are spending on projects versus day-to-day work can help answer the question about why so many projects are slipping their schedules, but first, you have to realize that projects are slipping their schedules, and second, you have to examine the reasons. It may not be the only reason. If you don't know there is a problem (or success) you won't look for a reason.

With all this in mind, I recommend combining two approaches:

1) Measure what matters to the people who hold you accountable. This might be the triple constraint. It might be ROI. It might be that a project actually completed (YAY!). If you deliver what your stakeholders value, your PMO and PMs will be more successful.
2) Use metrics that build on each other to drive the outcomes you are seeking. Don't just look at individual KPIs. Look at the big picture of how they work together to tell a story. Rather than telling you which metrics to use for this, I suggest that you examine the perceptions of your stakeholders. What do they say are problems with projects? What combination of measures will help you identify the root cause of the problems, when measured over time? Once you've identified the problem(s), continue to measure so you can tell if your response is improving the problem AND communicate this to help eliminate the perception that the problem might still exist.

Most importantly, emphasize the successes and ALWAYS have a plan for addressing any issues, when using KPIs to tell a story.
I was in charge of that. And I think some of the answers above are not quality related in my personal opinion. Quality have two make components: quality assurance (QA, it is on process) and quality control (QC,it is on deliverables). So, forget about the triangle, you have to get the strategic objectives and create the KPIs aligned to them. For example, one of the strategic objectives can be internal environment stability. Then you have to define the proccess inside your PMO that will impact that. When you define the process then you can define the KPIs based on process outcomes. Supposse your envionment involves technical environment. Supposse software products are a component inside your solution. Then, if inside your construction process you have a phase called UAT (User Acceptance Test) then you have to define QA and QC process to assure internal environment stability for UAT: it could be the exit criteria to get the GO to start moving the product to production after UAT. From the results of exit criteria then you can get KPIs.

Please login or join to reply

Content ID:

"It's kind of fun to do the impossible."

- Walt Disney