Milena IlievaProgram Manager Global accounts| VMWareVienna, Austria
Vijay,
The KPIs we measured (we are talking about large IT systems integration projects with external customers) were:
1. Finance (some of the metrics: revenue - contract to date (initial business case / actual); PM - contract to date (initial business case / actual); PM% - contract to date (...); Revenue- contract to go (initial business case / actual); PM - contract to go (initial business case / actual); P&L - year to date (IBC/budget/actual),...
2. Delivery &Technology - some metrics: total number of SLA to be met in the month; number of SLA met in the month; SLA % fulfilment; major incidents; number of opened incidents last month; number of opened incidents current month; incident backlog trend %; end user satisfaction; penalties amount; account improvement plan; offshore resources FTE (planned vs actual); productivity measurement (e.g. function points estimation), ...
3. Legal & Compliance - some metrics: contract signed; contract awareness; supplier and customer compliance obligations; complaints & claims; legal proceedings; data privacy; ..
4. Human resources - e.g. key staffing; planned FTE; actual FTE; team satisfaction, ...
5. Customer relationship - customer satisfaction reported based on the customer satisfaction questionnaire done at the beginning of the project, after major milestones + end of project
6. Suppliers & Partners - e.g. back to back agreement in place; supplier governance supplier quality; disputes & claims (supplier)
If you are interested, I can send you a file with the dashboard with complete list of KPIs and measurements. Of course each project would report what is applicable and according to the specifics.
I hope this helps.
Regards,
Milena
...
1 reply by Vijay Gopinathan Pillai
Sep 03, 2020 11:59 PM
Vijay Gopinathan Pillai
...
Thank Milena for the detailed insight as i can relate to these type of large integration projects. Would be great if you could send me the dashboard file , would help in getting a better overview.
Awesome Milena! I believe our annual departmental results report only has two KPI's/metrics.
1. Planned/approved budget vs actual 2. FTE planned vs actual
Of course, there is much more reporting going on below the surface, but the minister's office thought that just those two KPI's were sufficient. For example, our assistant deputy minister is often asked to report on how much money is spent vs planned on business travel. In fact, projects must plan for this funding line every year amongst many other financial commitments such as civilian salaries, military salaries, employee benefits, employee training costs, overhead, etc.
...
1 reply by Vijay Gopinathan Pillai
Sep 04, 2020 12:06 AM
Vijay Gopinathan Pillai
...
Correct Steve , typically organisations put KPIs mostly on Cost and Schedules on priority as Keith mentioned too. My idea was to also bring this topic to the table so that all could give their valuable inputs with respect to their organisation and it could allow others to check their relevance with respect to their organisation then.
I've seen lots of different KPIs in my different job roles related to aerospace.
Often the technical performance like payload vs. unit cost will determine the market share for the eventual product, so those wind up being critical metrics to a profitable program.
When there is a project to develop a new product where each customer needs a customized deliverable, the metrics include not only the original project developing the first unit, but also the expected recurring cost for the labor to customize each new customer's product.
In system design projects, there are often many dependencies between teams. A common KPI would be the % of identified vs. negotiated interfaces/data exchanges because late interface data = rework for affected teams.
In a manufacturing environment where a product moves from one place to another in the production facility, the amount of work left incomplete when the product moves from one line position to another is a metric of manufacturing disruption.
In cost improvement projects, the ROI is often used to determine how efficient we are using our time. There may be another metric for how much we thought we were going to save when we launched a product, vs. how much we actually saved in order to evaluate our estimating ability.
When trying to grow, or reduce various parts of the business, we may have a KPI related to our strategic goals like the amount of work sent to a specific business area.
An example of how they may differ between agile and predictive approaches is on time performance, vs. rework. In my very first engineering job, we waited until the very last minute to complete our deliverables to reduce the opportunity for some change to drive rework. Our on-time performance suffered, but we rarely required rework. In my next job, we released deliverables as early as possible. Our on-time performance was great, but we had to revise most deliverable several times at significant expense.
It is said that what gets measured, gets done, so while KPIs may seem more theoretical, they very much can drive behaviors, and not always the behaviors that are best for the business.
...
1 reply by Vijay Gopinathan Pillai
Sep 04, 2020 12:22 AM
Vijay Gopinathan Pillai
...
Absolutely rightly said , whats gets measured , gets done. The ones mentioned by you here do rightly fall in the place to ensure the program/project is properly measured and so eventually properly meets the done criteria. The shift from a theoretical to practical usage could be the norm moving forward.
The KPIs we measured (we are talking about large IT systems integration projects with external customers) were:
1. Finance (some of the metrics: revenue - contract to date (initial business case / actual); PM - contract to date (initial business case / actual); PM% - contract to date (...); Revenue- contract to go (initial business case / actual); PM - contract to go (initial business case / actual); P&L - year to date (IBC/budget/actual),...
2. Delivery &Technology - some metrics: total number of SLA to be met in the month; number of SLA met in the month; SLA % fulfilment; major incidents; number of opened incidents last month; number of opened incidents current month; incident backlog trend %; end user satisfaction; penalties amount; account improvement plan; offshore resources FTE (planned vs actual); productivity measurement (e.g. function points estimation), ...
3. Legal & Compliance - some metrics: contract signed; contract awareness; supplier and customer compliance obligations; complaints & claims; legal proceedings; data privacy; ..
4. Human resources - e.g. key staffing; planned FTE; actual FTE; team satisfaction, ...
5. Customer relationship - customer satisfaction reported based on the customer satisfaction questionnaire done at the beginning of the project, after major milestones + end of project
6. Suppliers & Partners - e.g. back to back agreement in place; supplier governance supplier quality; disputes & claims (supplier)
If you are interested, I can send you a file with the dashboard with complete list of KPIs and measurements. Of course each project would report what is applicable and according to the specifics.
I hope this helps.
Regards,
Milena
Thank Milena for the detailed insight as i can relate to these type of large integration projects. Would be great if you could send me the dashboard file , would help in getting a better overview. Saving Changes...
Awesome Milena! I believe our annual departmental results report only has two KPI's/metrics.
1. Planned/approved budget vs actual 2. FTE planned vs actual
Of course, there is much more reporting going on below the surface, but the minister's office thought that just those two KPI's were sufficient. For example, our assistant deputy minister is often asked to report on how much money is spent vs planned on business travel. In fact, projects must plan for this funding line every year amongst many other financial commitments such as civilian salaries, military salaries, employee benefits, employee training costs, overhead, etc.
Correct Steve , typically organisations put KPIs mostly on Cost and Schedules on priority as Keith mentioned too. My idea was to also bring this topic to the table so that all could give their valuable inputs with respect to their organisation and it could allow others to check their relevance with respect to their organisation then. Saving Changes...
I've seen lots of different KPIs in my different job roles related to aerospace.
Often the technical performance like payload vs. unit cost will determine the market share for the eventual product, so those wind up being critical metrics to a profitable program.
When there is a project to develop a new product where each customer needs a customized deliverable, the metrics include not only the original project developing the first unit, but also the expected recurring cost for the labor to customize each new customer's product.
In system design projects, there are often many dependencies between teams. A common KPI would be the % of identified vs. negotiated interfaces/data exchanges because late interface data = rework for affected teams.
In a manufacturing environment where a product moves from one place to another in the production facility, the amount of work left incomplete when the product moves from one line position to another is a metric of manufacturing disruption.
In cost improvement projects, the ROI is often used to determine how efficient we are using our time. There may be another metric for how much we thought we were going to save when we launched a product, vs. how much we actually saved in order to evaluate our estimating ability.
When trying to grow, or reduce various parts of the business, we may have a KPI related to our strategic goals like the amount of work sent to a specific business area.
An example of how they may differ between agile and predictive approaches is on time performance, vs. rework. In my very first engineering job, we waited until the very last minute to complete our deliverables to reduce the opportunity for some change to drive rework. Our on-time performance suffered, but we rarely required rework. In my next job, we released deliverables as early as possible. Our on-time performance was great, but we had to revise most deliverable several times at significant expense.
It is said that what gets measured, gets done, so while KPIs may seem more theoretical, they very much can drive behaviors, and not always the behaviors that are best for the business.
Absolutely rightly said , whats gets measured , gets done. The ones mentioned by you here do rightly fall in the place to ensure the program/project is properly measured and so eventually properly meets the done criteria. The shift from a theoretical to practical usage could be the norm moving forward. Saving Changes...
Wade HarshmanScrum Master| GDITIndianapolis, In, United States