Project Management

Project Management Central

Please login or join to subscribe to this thread

Topics: Agile, Benefits Realization, Using PMI Standards
how are measurement of Metrics and KPIs accurate
Sort By:
The DOE helps in analysing various almost complete scenarios of your KPIs and metrics. control charts give the necessary inputs for it. and wala a fine tuned good practices emerges with rules, policies and procedures in place which will lead to standardization in industry.
...
1 reply by deepak snraj
Nov 07, 2022 6:04 AM
deepak snraj
...
Accuracy is whether you are achieving the intended result. For measurements, that is whether your measurement reflects the right thing. To evaluate the accuracy of some measurement scientifically, you either need to compare the measurement against some known value (control) or you must understand the underlying physics well enough to be confident your measurement actually measures your intended quality or quantity.

Control charts and fine tuning are often used to evaluate precision (repeatability) rather than accuracy. I could measure customer satisfaction with software by counting lines of code with more code = better, and count bits of code to get an even more precise answer. Neither tell me anything about customer satisfaction (accuracy).

I could ask the customer if they are satisfied (Y/N) and get a more accurate but imprecise answer. I could give the customer a survey that never once asks about their satisfaction but seeks to measure qualities they desire to get a more precise answer but is it accurate? To evaluate the accuracy I could test a control group against the survey by comparing known satisfaction level against the survey output.

Some KPIs are hard to test, in which case you need to ask “What am I really measuring?” and evaluate whether the KPI is merely correlated to the thing you wish to measure, or causally related.
To your original question, I would suggest that independent verification, calibration against eatablished measures and parallel, alternate measurements will help you assess KPI accuracy.

Kiron
I might be missing something, but I don’t see the original question?
That question should be asked when you set up your KPIs. KPIs need to be objective rather than subjective. If you can't measure it it isn't a KPI, its an opinion.
Think KPIs as a quality management process - percentage of time or units one has achieved delivery withing established tolerances.
Accuracy is whether you are achieving the intended result. For measurements, that is whether your measurement reflects the right thing. To evaluate the accuracy of some measurement scientifically, you either need to compare the measurement against some known value (control) or you must understand the underlying physics well enough to be confident your measurement actually measures your intended quality or quantity.

Control charts and fine tuning are often used to evaluate precision (repeatability) rather than accuracy. I could measure customer satisfaction with software by counting lines of code with more code = better, and count bits of code to get an even more precise answer. Neither tell me anything about customer satisfaction (accuracy).

I could ask the customer if they are satisfied (Y/N) and get a more accurate but imprecise answer. I could give the customer a survey that never once asks about their satisfaction but seeks to measure qualities they desire to get a more precise answer but is it accurate? To evaluate the accuracy I could test a control group against the survey by comparing known satisfaction level against the survey output.

Some KPIs are hard to test, in which case you need to ask “What am I really measuring?” and evaluate whether the KPI is merely correlated to the thing you wish to measure, or causally related.
Oct 31, 2022 5:35 AM
Replying to deepak snraj
...
The DOE helps in analysing various almost complete scenarios of your KPIs and metrics. control charts give the necessary inputs for it. and wala a fine tuned good practices emerges with rules, policies and procedures in place which will lead to standardization in industry.
Accuracy is whether you are achieving the intended result. For measurements, that is whether your measurement reflects the right thing. To evaluate the accuracy of some measurement scientifically, you either need to compare the measurement against some known value (control) or you must understand the underlying physics well enough to be confident your measurement actually measures your intended quality or quantity.

Control charts and fine tuning are often used to evaluate precision (repeatability) rather than accuracy. I could measure customer satisfaction with software by counting lines of code with more code = better, and count bits of code to get an even more precise answer. Neither tell me anything about customer satisfaction (accuracy).

I could ask the customer if they are satisfied (Y/N) and get a more accurate but imprecise answer. I could give the customer a survey that never once asks about their satisfaction but seeks to measure qualities they desire to get a more precise answer but is it accurate? To evaluate the accuracy I could test a control group against the survey by comparing known satisfaction level against the survey output.

Some KPIs are hard to test, in which case you need to ask “What am I really measuring?” and evaluate whether the KPI is merely correlated to the thing you wish to measure, or causally related.

Please login or join to reply

Content ID:
ADVERTISEMENTS

It is wonderful to be here in the great state of Chicago.

- Dan Quayle

ADVERTISEMENT

Sponsors