by Laurent Thomas, PMBOK® Guide-Seventh Edition Development Team member
As project leaders we have to understand where we are and, with the help of the stakeholders, take decisions to keep project work on track. Measures, most often numbers, are the usual way to identify and communicate the current state of the project, and upon which to base discussions and decisions. Unfortunately, numbers can be deceptive. Pitfalls in selecting the informative measures, errors in analyzing them, misinterpretation when communicating them can lead to less than optimal decisions.
Cognitive biases, misapplication of statistics, cheating (gaming the numbers), or simply plain ignorance are all impediments to the benefits of using measures. Finding the relevant metrics, the right way to measure, to analyze, to understand and interpret their variance, and finally communicate metrics require a set of skills that every project team must possess.
There must be agreement between the project team and organizational leaders or governance associated with the metrics and their associated measures. But it is also important to consider who will use specific measures. Consider velocity. This metric is an internal team measure to help the team consider ways to improve its performance. But this measure is not intended for sharing with external stakeholders.
Measurement, whether collecting, analyzing or communicating metrics, is considered an acquired skill. Every project team member and stakeholder is supposed to know how to deal with numbers. But one thing to consider is that a measure may not be necessarily a number. For instance, using letters rather than numbers to select an answer in a customer satisfaction survey would indicate more clearly that average is a meaningless representation of the centrality of the responses.
As project leaders, one of our (numerous) responsibilities is to correctly assess the situation based on free-from-bias observations and interpretations, present it in a non-equivocal manner, and help the team take the right decision based on a rational analysis. Not a simple feat. All hope is not lost, however. To reach a reliable understanding of the status of the project or facilitate decision making, project team members and project leaders alike must improve their ability to grasp the consequences of our human cognitive biases and limited statistical skills. We must acquire the knowledge and capabilities that will help us navigate the complexity of number crunching. For instance, mastering the difference between causation and correlation is not out of reach, nor is identifying a Simpson paradox occurrence in a set of project data. Making sense of p-value will certainly help projects in forecasting the expected level of the quality of the next release.
Measurement is necessary for every project, and measurement done right is useful and informative. However, we should not forget that measures are not the ultimate aim of a project; but a means to reach a much more important goal: delivering business value.
Correctly interpreting measures is thus an essential skill for understanding elements of project work and has direct bearing on the ultimate outcome of the project. This skill must and can be learnt. It is critical to know what to measure, when to measure, and how to interpret and present the measure without falling prey to the cognitive biases that lead to distortion or illogical interpretation.
Therefore, I believe measurement should be considered as one of the Performance Domains for project success for any project. And I strongly urge my fellow project leaders to master basic statistics and be aware of our human brain deviation from rationality. By doing so we will rely upon sound project status to help our projects achieve their targets and generate their expected business value.