3 Principles for Project Metrics
On your project you will have a number of measures and metrics. Key performance indicators (KPIs), critical success factors (CSFs) or success criteria and benefits can all be measured, and you may have some other targets to achieve as well such as customer satisfaction measures.
So what makes a good metric?
According to John Seddon in Freedom from Command and Control: A better way to make the work work, there are 3 principles for making sure your metrics are robust and useful.
1. Does it help understanding?
Seddon believes that the test of a good measure is that it helps us understand and improve performance. If it doesn’t, there isn’t much point in gathering the data. Targets, for example, are measures that don’t help us understand performance. They are arbitrary figures, created perhaps on a whim. Worse, they can focus people on the wrong things, and not on how best to do the work.
Capability measures look at the system of work (in a systems thinking environment), so they encourage people to look at how things are done and therefore they help improve the work. They can be more motivating than arbitrary targets.
In a project setting, by this definition a target held by the Project Management Office of closing 10 projects each quarter is not a good metric. A measure of how long it takes to complete the testing phase is useful, as that gives you data you can compare across a number of projects. Then you can draw some conclusions as to why some projects have a longer testing phase than others and in turn this will help with your estimating.
2. Does it relate to purpose?
The second test of a good measure, according to Seddon, is that the measure must relate to “the purpose of an organisation from the customers’ point of view”. In other words, measures must relate to something that means something to someone.
In a project environment, the person who is likely to be deciding what is important is your main project stakeholder or sponsor. Whatever matters to them is the thing you should be focusing on. For a project manager, this could be the triple constraint in the form of the time-cost-quality triangle. Or you could be told that the response times on the new website you are building are the most important thing and the sponsor wants reports every month about how much you have been able to improve them by. A lot of project measurements don’t actually relate back to what internal and external customers care about, so it is worth looking at what your project is measuring and checking that you aren’t wasting your time.
For more about what customers care about and how you can identify what is important to them, my book, Customer-Centric Project Management, includes a couple of case studies and some templates.
3. Does it integrate with the work?
The third test of a good measure is that it is easy enough to record. If it doesn’t integrate with the way that the work is done, no one will collect the data. That’s true enough – I’m sure you have occasionally been asked to produce reports and have struggled to find a way to represent the data or even to find it, as it isn’t ‘natural’ to be capturing project information in that way.
Seddon also believes that metrics that look backwards aren’t much help. Making decisions based on data that was captured in the past isn’t a good way to plan work going forward. I don’t agree – in a project environment capturing lessons learned, time measures and data that will help future estimates is, in my opinion, a good use of time. But I do see what he means in a service environment. His book is aimed at people in a call centre-type service organisation or manufacturing, where knowing that last week you answered 30 calls an hour or built 60 widgets doesn’t give you an accurate prediction of what will happen this week.
How many of your project measurements meet Seddon’s 3 criteria? And if they don’t adhere to these principles, are you happy about why not?
Business case basics (part 2)
Last time I looked at some of the basic features of the project business case – the key document that sets out what the project is going to achieve, how the work will be done, and most importantly, why this is a problem that needs to be solved right now. I covered the purpose of a business case, who is likely to be reading it, and the general format. Today I want to look at what different sections go into the document to make up the full business case.
These aren’t in any particular order, and you can arrange the segments as you like, perhaps according to whatever template is used as standard by your Project Management Office. However, if you are going to use an executive summary, that has to come at the beginning, so let’s look at that part first.
Normally, you write the executive summary last. It is a short statement, normally no longer than a page, that sets out a summary of the whole document. It is for people who don’t have time to read the whole thing, but it is also useful for everyone. You get a complete flavour for what the document will be about and how the project is likely to unfold, and then you can read the detail on the subsequent pages.
Because it is a summary of the detail of the document, that’s the reason we write it last. You need to know what goes in to the rest of the document before you can summarise it here!
Statement of need or problem
Early on in the business case you should set out what the problem is that this project will address. After all, readers will quickly want to know more about the issues faced by the company and how this project will fix them. Mention the current situation, if this project relates to improving a particular system or process. Figures and other types of data, such as customer surveys, can also be useful for making the point that there is a real issue that needs addressing. If you can, link the problem back to strategic objectives or key performance indicators for the company, maybe referring to elements of the balanced scorecard.
The important thing in this section is to remember that it will be business people who are reading it, not technical staff or project managers. Try to avoid jargon and make sure that you set out the problem and solution clearly.
A business case should look at all the potential ways to remedy this problem before making a recommendation on the best one, so include a section where you spell out the different options. By this point you’ll already have a view on which solution you will be recommending, so don’t spend too much time describing options that have been rejected and why they are no good. However, you should mention them, so that the readers know the thought process that you went through to identify the best way forward.
Remember that there is always an option to do nothing, so you should also include this as a comparison to the proactive options.
In this section, spend more time explaining your proposed solution. You do have more space to go into detail, but again remember to keep it all in business language avoiding jargon where you can and being really clear about how this particular solution will help solve the problem.
This section is something that you could refer to later when putting together your project initiation document or charter, so it can also include a summary of costs and benefits, key milestone dates and who will be required to work on the project. If you don’t know names at this point, make a note of the key skills or job titles required.
Every business case needs a section on costs as this is one of the main ways that people make decisions about what projects to do and how to prioritise them. Costs cover a number of different areas so consider these:
You can probably think of some others yourself.
The flip side of costs is benefits, and your business case should make it clear what benefits will be achieved by working on this project. Benefits could include things like increase in customers, revenue, staff morale, or avoiding costs, or ensuring the company remains compliant with current legislation.
This section covers positive and negative risk and you would want to include some detail about what mitigation plans would need to be put in place to manage the risk if the project was approved. You can group risks into categories if that makes it easier to present the information to the readers. Your Project Management Office may already have a standard set of categories, but if not you could consider things like technical risk, business risk, and any dependencies on other areas or projects, as these could also be risks.
Those are the main considerations to look into when producing your business case, although there are no hard and fast rules and it is best to include more information (as long as it is clear and concise) than less. This gives people the information they need to be able to make an informed decision about whether to progress with the project.
Business case basics (part 1)
One of the things that project managers get involved with from time to time is preparing business cases. Even if you don’t have to write the business case from scratch, there is a possibility that you will be asked to put some of the data together, or to review the final version. And, of course, at the end of the project you’ll have to go back and compare the project results to what the business case said, so it is a useful document to understand. You can keep a copy with your project files.
Here is some more information about this important project document.
Why bother with a business case?
There are three main reasons why you should make sure that your project has a business case (even if it isn’t written by you). They are:
Essentially, a business case answers the following questions:
What? What are we doing? What problem are we trying to solve?
Why? Why is this a particular issue right now? Why should we bother working on it?
How? How will we fix the problem? How will the work be carried out?
Who reads the business case?
It’s important to think through who is the target audience for a business case if you are writing one or providing information to go into one. The business case will end up being used by a number of different people or groups, so it should include information at different levels to suit their varying needs.
Think about who will read it:
What format does the business case take?
Typically project business cases are documents. Even today, when so much of our project work is done online and via a range of different tools that support project planning and document sharing, the business case is still a document that is formally put together in its own right, outside any project collaboration software. You can still upload the business case to your document store, if the project goes ahead, but at this point you are probably looking at putting together a document.
The document is likely to include links to or an embedded spreadsheet, or this could be included as an appendix. The spreadsheet elements cover things like more detailed assessments of costs and benefits, and maybe a resource plan. You could also include screenshots from project planning software if your business case includes high level milestones or any scheduling information. This can be useful so that decision makers can see the overall length of time that the project is planned to run for, and make their decision based on how long resources will be occupied working on the project.
Next time I’ll be looking at the major elements that go in to the document itself. In the meantime, who reads the business cases on your projects? Let us know in the comments.
2 Types of Measures
“There are two kinds of measures of use in managing a system: permanent and temporary,” writes John Seddon in Freedom from Command & Control: A better way to make the work work. “Permanent measures, those that are related to purpose, are the guiding measures for all performance-improvement work. Temporary measures are those that are useful in ascertaining the nature and size of a problem before action is taken.”
Seddon is an occupational psychologist who has spent his career trying to improve systems by taking the waste out of processes. One of the ways he has done this is by focusing on what gets measured, a kind of ‘What gets measured gets done’ approach to improvement. His work has no doubt resulted in a lot of projects, particularly those that relate to continuous process improvement, but his book doesn’t cover project environments particularly. Still, I think that the way he splits measures into these two types is relevant to project managers – and we are focusing on governance this month at ProjectManagement.com. Let’s take a closer look at the types of measures Seddon explores in his book.
Permanent measures, are, as you have probably guessed by the name, permanent. As nothing much is permanent on a project, these are the kind of measures you are likely to find in your Project Management Office or Portfolio Office. They are the measures that help manage the work overall. Some examples of permanent measures are:
Capacity and demand. This can help with resource scheduling on an ongoing basis. It helps to know what work is due to complete and when project managers will be free to take on other projects.
End-to-end time. This is how long it takes to complete the work from the customer’s perspective. As not many projects are repetitive, this is a bit of a pointless measure on projects. However, if you are repeating similar initiatives over a period of time (such as rolling out new software to an office at a time), it can be very useful to know how long it takes to complete one implementation.
Accuracy. In a project environment, this would be covered by project audits. The ‘accuracy’ of a project could equate to scope items covered, fit with requirements or how customer-centric you are and how much value you create for customers.
Temporary measures are those that don’t last long. They aren’t permanently built into the fabric of how companies report projects. As such, you could have temporary measures on your project, especially if your project has a RAG status of Red and is in trouble. If you are trying to rescue a failing project, you will want to gather some data on what has been done to date and how best you can recover the situation.
It’s less easy to see how Seddon’s examples of temporary measures relate to projects. Here they are.
Type and frequency of demand. This could be a PMO measure, especially if you are looking to recruit an additional project manager or more staff in other capacities. You could measure the type and frequency of demand over a short period of time to check that you really do have cause to hire someone. Equally, on your project, you could use this type of metric for resource management. For example, as you move into the testing phase your resource needs will change. This is something that you won’t measure when the project is over as staffing up to support the product is a task for the operational team.
Type and frequency of ‘dirt’ in input. Seddon is mainly writing about service organisations like call centres, so he means incomplete application forms, letters that don’t have enough information in and so on. In the world of projects, this could be things like project initiation forms that don’t have enough information in for the stakeholders to sign off the scope, incomplete business cases, requests for projects that appear in the form on an email with a good idea rather than a thought-through assessment for a new project etc. He recommends that a few small changes could make a big difference, such as changing the forms or template letters used. For example, you could amend the form on your company intranet so that it is rejected unless someone enters all the detail required to submit a new project for consideration.
Type and frequency of waste. ‘Waste’ to Seddon means steps in the process that don’t add any value. These could be things like handovers between teams, lost time, rework and errors. There are probably numerous places on a project or in your project management processes where things get lost or handoffs between teams are not as slick as they could be.
I’m a big believer in taking what we can from other areas of management practice and applying them to projects, so I think it is worth considering what we can learn from the type of metrics used in service organisations and the like. Do your project or PMO metrics fit any of these categories?
What is bottom up estimating?
In this installment of my series on estimating, I look at what bottom up estimating is in a short video.