Categories: Metrics
Benchmarking is a hot topic in the "waterfall" world. I had the chance to work on a process improvement project with people that contributed to the ISO standards. It is a fascinanting world: the world of Function Points. Like in standard Project Management there are 2 complementary groups: the IFPUG and COSMIC. They are in a healthy competition and in my view complementary rather than opposite.
I see in many Agile forums questions on metrics and I sense the fear of good metrics. It looks like everybody wants to prove that Agile is great, that when they moved to Agile it was a smart choice and everything is delivered faster and cheaper. I know two groups that claim that using their approach projects can deliver double scope in half the time. Neither has proof that it can be done. From a Lean Six Sigma perspective it is practically impossible to have a 10-15% improvement, unless what you did before was extremely inefficient. Even in that case, if the team was that inefficient and didn't know it it is unlikely that adopting Agile the situation will change.
Teams try to compare the "velocity" using Story Points and that's how the gaming starts: splitting backlog items, changing the number of story points during the sprint or inflating them at the Sprint Planning.
I haven't heard yet of someone proving that an adaptive approach is more efficient that a planned approach. Stores like we failed the project first with "waterfall" then we delivered the scope using Agile need detail. Maybe the success of the second attempt was built on the lessons learned from the failure. I put few projects on track moving from "Scrum" to PMBoK. It worked because the organisation and the project team were not ready for Agile and the move to Agile was an excuse for not understanding their own business. They heard that in Agile there is no need of documentation, they can change their mind every month etc. but they didn't know that all those "benefits" come at a cost and with the risk that after 4 months you can be in the same point as when you started.
Is Benchmarking possible or relevant in Agile. I believe that the answer is yes. Agile is (or it should be) a process improvement initiative: "better ways" but we need metrics to prove that we are on the right track. If the organisation is really Agile there will be trust and the project team won't game the metrics. But that is rare, especially when you are under pressure to prove that Agile works.
My recommendation is that organisations should baseline their delivery process before jumping into the Agil train. Metrics like the time from the request to actually working on it, from the request to production release. Organisation can also baseline the overhead required and that's where things may get complicated :).
In a traditional project, called "waterfall" the overhead is the cost of the PM, BA, testing and deployment. Can Agile build cover the lack of detailed requirements? Can the team self organise to a point where there is no need for a manager? is less or no testing done? Is the deployment process faster and cheaper?
If the answer is predominantly "no", then the organisation may not be ready for Agile.
