Have you encountered situations where averages or advanced analytics were applied to ordinal metrics like story points or Planning Poker estimates?

Agile Estimating Ethics

Stelian ROMAN Project Manager| MicroSafety Carlingford, New South Wales, Australia

In Agile software development, metrics like Planning Poker story points are widely used to estimate the size and complexity of work items. These metrics are based on ordinal scales—a type of ranking where the relative order of items matters, but the exact differences between them do not. Despite this, it’s common practice to calculate averages, run regressions, and otherwise apply standard mathematical operations to such data. This statistical misuse isn’t just a technical mistake; it has real-world consequences for decision-making and can cross into the realm of ethical misrepresentation. In this blog post, we examine the nature of ordinal data, why treating it as interval data is problematic, and the ethical implications for teams and organizations. We also provide guidance to help avoid these pitfalls, concluding with a question for readers to reflect on their own experiences.

Have you encountered situations where averages or advanced analytics were applied to ordinal metrics like story points or Planning Poker estimates?

How did it affect planning, transparency, or trust in your teams?

Blog post ProjectManagement.com - Statistical Misuse of Ordinal Scales: The Mathematical and Ethical Flaws of Averaging Planning Poker Metrics

Posted: Jun 15, 2026 1:26 AM (Updated by author: Jun 15, 2026 1:29 AM)

Sort By:

Luis Branco CEO| Business Insight, Consultores de Gestão, Ldª Carcavelos, Lisboa, Portugal

I have encountered situations where increasingly sophisticated analyses were applied to story point data.

The most significant impact was not the statistical output itself, but the confidence that those outputs created in planning and forecasting discussions.

When estimates are treated as precise measurements, organizations can gradually confuse precision with understanding.

At that point, the discussion shifts from exploring uncertainty to defending numbers.

A valuable reminder that the quality of decisions depends not only on the quality of the analysis, but also on a clear understanding of the limits of the data being analysed.

...

1 reply by Stelian ROMAN

Sep 07, 2026 10:18 PM

Stelian ROMAN

...

Excellent point. Not only that (some) people must learn the meaning of estimation, but I've worked with very few Agile teams that understand what relative estimation is or what semiquantitative metrics are.
When you assign a 3 or a 1 it doesn't mean that the item scored with 3 will take 3 times more. A 1 may require 1 hr, while a 3 can take a couple of days. I personally find the addition of story points misleading and dangerous outside of Sprint planning. I've seen a multimillion-dollar program 'managed' using story points. Should I mention that it was a disaster?

Posted: Jun 15, 2026 4:34 AM (Updated by author: Jun 15, 2026 4:35 AM)

Stelian ROMAN Project Manager| MicroSafety Carlingford, New South Wales, Australia

Jun 15, 2026 4:34 AM

Replying to Luis Branco

...

I have encountered situations where increasingly sophisticated analyses were applied to story point data.

The most significant impact was not the statistical output itself, but the confidence that those outputs created in planning and forecasting discussions.

When estimates are treated as precise measurements, organizations can gradually confuse precision with understanding.

At that point, the discussion shifts from exploring uncertainty to defending numbers.

A valuable reminder that the quality of decisions depends not only on the quality of the analysis, but also on a clear understanding of the limits of the data being analysed.

Posted: Sep 7, 2026 10:18 PM

Please login or join to reply

Have you encountered situations where averages or advanced analytics were applied to ordinal metrics like story points or Planning Poker estimates?

Sponsors

Newsletters