Viewing Posts by Kevin Korterud
Award-Winning Metrics For 2018
by Kevin Korterud
What are the best metrics for determining if a project is about to experience schedule, budget or quality slippages? These metrics are best categorized as delivery volatility metrics.
Executives already know when a project is in trouble — they are more concerned with those projects whose trajectory is on a currently unseen course to trouble.
PMI offers guidance on project metrics to help detect delivery volatility, such as the Cost Performance Indicator and Earned Value Management. While project reporting will likely have one or more of these metrics, I got to thinking what other metrics would indicate the potential of delivery volatility.
An additional complication is the various approaches used today, including agile, waterfall, company custom, software product, service supplier and regulatory. These can all generate their own set of metrics.
While pondering this question watching TV one evening, I noticed a multitude of movie, theater, television and music award shows that tend to occur this time of year. A characteristic of these shows is the numerous categories that are awarded to nominees — Best Supporting Actress, Best New Pop Group, Best Special Effects and so on.
As I was organizing my thoughts around metrics, I figured: Why not use award show categories to help shape an answer on which metrics would best suit early detection of delivery volatility?
As the Master Of Ceremonies for the 2018 Project Metrics Award show, here are a few of the winners:
As our projects become more complex and more numerous, the ability to deliver on a set schedule becomes more important. The SPI has the great benefit of comparing actual and planned progress in an objective manner: earned value/planned value.
The true power of SPI comes into play when selecting a method for earned value accumulation. Assuming work plans are at a level of granularity where task progress can be measured within a two to four week window, a conservative earned value scheme such as 0%/100%, 25%/75% based on task start and completion is a very objective means of calculating progress.
With these conservative schemes, you capture value when the tasks have started (when resources are truly free to work on tasks) and whether the task has been completed (usually with acceptance of completion by a project manager or stakeholder).
Given today’s tight delivery timeframes, as well as the need to coordinate delivery with other projects, SPI is a good indicator as to the schedule fitness of a project.
2. Best Supporting Emerging Metric: Functional Progress Metrics!
As I shared above, there are now a multitude of methods available to run projects. From these methods, all sorts of new metrics are available to project managers to identify delivery volatility. These metrics can include completed user stories, forecast backlog, project burndown, build objects, test case performance and many others.
In addition to these new metrics, a whole host of new waterfall, agile and other tools have come into play that capture functional progress outside of the traditional work plan tasks and milestones. In fact, work plan detail requirements can be relaxed when these tools are used to shed light on the functional progress of a project.
The power of these functional metrics is that they allow the next level of inspection underlying project phases, tasks and milestones to see delivery trajectory. For example, being able to see the detailed completion progress of requirements, build objects and test cases in automated tools allows project managers to catch underlying barriers to progress before it is revealed in a work plan.
As project managers, the universal outcome for our efforts is that we need to create value for our project executives and stakeholders. While activities can lead to creating value, our mission revolves around the production of deliverables in a timely manner to fulfill a project value proposition.
The inherent power in providing and approving deliverables in a timely manner is that they are completely objective means of progress. No matter what method, effort, dependencies, resources, tools or other constructs of project management are employed, deliverables are an indicator of whether you are making progress. The track of deliverables being created, reviewed and approved on schedule means you are making definitive progress toward value.
Creating a track of deliverables and their targeted completion dates with progress that can be monitored through other metrics allows a universally understood path to project completion. For example, if a deliverable has not yet been approved by stakeholders, you are making visible a potential schedule delay that would impair future work activities.
To host your own 2018 project metrics award show, one does not need a spotlight or trophies. You just need to think about what metrics can serve to detect early signs of delivery volatility beyond the self-declared green/yellow/red stoplights that are typically found in project status reports.
If you were handing out your very own 2018 project metrics awards, what categories would you select? What would win?
I’m frequently asked for insights on performance measurement criteria for project managers. This comes as a bit of a surprise given how professional certification programs, such as PMI’s Project Management Professional (PMP®) certification, have brought more consistency to project management skills.
Organizations’ typical performance measurement framework for functional roles is focused on growth and results. But that framework is becoming less effective at measuring project managers.
Project managers differ from functional roles in that they perform their duties with definitive time periods, outside influences, ever-changing activities and a higher level of uncertainty.
At the same time, more and more companies are seeking both individual and aggregate project management performance measures. Aggregate measures provide insights into overall capabilities and indicate if improvement initiatives — training, methods, processes — are actually increasing project manager productivity.
I’ve spent some time thinking about how to improve measurement criteria for project manager performance. Here are three areas I believe must be included:
Over time, individual project manager metrics, such as schedule and budget, can be analyzed to show the project manager’s track record. Supplementary metrics, such as change control activity, deliverable finish date delays and cost of poor quality, can provide a complete picture of project manager performance.
By aggregating and averaging these metrics — as well as using other data points such as labor cost — the enterprise capability of project managers can be measured.
2. Project Manager Engagement Reviews: The ability of a project manager to successfully engage with stakeholders is a key success factor for projects. A high level of engagement allows for early visibility to potential delivery issues, as well as a stronger understanding of the success criteria for a project.
The most effective means to measure project engagement is to conduct a post-project review with the project’s primary stakeholder. As engagement is not a binary yes/no condition, open-ended questions allow for deeper insights into the project manager’s level of engagement. For example, probing when project managers anticipated potential project issues would help to reveal engagement. These reviews are not meant to be punitive, but instead to guide and educate.
In addition, the reviewer should also look at the engagement level of the primary stakeholder. It’s not uncommon to find unengaged stakeholders, which can lead to poor delivery results for which the project manager is unfairly held to account. A balanced view of both the project manager and stakeholder will give the reviewer a true measure of engagement.
Capturing project performance data allows project managers to share successes, as well as provide rationale for when things might not have gone as well as anticipated. It serves as a platform for career growth.
In today’s world of ever-increasing project complexity and scale, both companies and project managers need to expand their demonstrated performance results beyond what is found today.
How do you measure project manager performance? Do traditional performance measurement frameworks for functional roles continue to meet the need?
By Kevin Korterud
I experienced my first agile project nearly a decade ago. At the time, agile was still an emerging concept. I remember thinking there were all sorts of activities going on that I had never seen on any of my projects. People were standing up for meetings, marker boards were filled with things called “stories” and delivery moved forward under the framework of a “sprint.”
At the center of this whirl of frenetic activity was a person who the team called a “scrum master.” At first, I thought this person was a project manager. But they were doing things that were outside of the traditional project management realm.
Since that first experience, agile has matured and continued to grow in popularity. This trend prompted me to examine the evolving role of the scrum master in complex agile delivery environments. Here are my observations:
1. Agile Delivery is Becoming Mature
Agile delivery teams used to function within isolated pockets. But, as the use of agile—as well as the size and complexity of solutions being delivered—grew, new methods, such as SAFe®, were developed to help orchestrate agile delivery across an organization.
With agile becoming more common in organizations as a delivery method, the overall need for scrum masters’ general process advice diminishes. Agile teams over time—as well as with the support of enterprise framework methods—will become more self-sufficient, which reduces the need for some of the current activities performed by scrum masters.
2. Higher Engagement and Direct Accountability
One of the guiding principles for scrum masters is that they are not supposed to intervene with the team and are not responsible for delivery outcomes.
While a focus on process advice was essential during the early days of agile, today’s larger and more complex solutions demand that delivery quality issues be identified as soon as possible. In addition, there is also a need to ensure on a more frequent basis that the solution being created will yield the desired business outcomes.
Given its proximity to agile delivery teams, the scrum master role is positioned to leverage a higher level of engagement and accountability. In addition to traditional agile process advice, scrum masters should also serve as a durable checkpoint for both delivery quality and alignment to business outcomes.
These checkpoint activities would include reviewing user story quality, monitoring non-functional requirements and checking solution designs against business needs. As other roles in agile delivery possess some form of delivery accountability, the scrum master must also become more engaged and accountable in order to remain relevant.
3. Emerging Project Managers Becoming Scrum Masters
While scrum masters are not meant to be project managers, that notion is preventing project managers from becoming scrum masters, especially earlier in their career. Emerging project managers invariably have some form of solution delivery experience. They know what makes for sound requirements (especially non-functional), designs, testing, quality and implementation plans.
As the level of complexity and scale increases with agile delivery, so does the need for some form of delivery oversight at the agile team level. With the scrum master position in their repertoire, teams would have developed competencies and know-how for scaled agile delivery, release train engineer, program manager, etc.
Scrum masters have played an essential role in the growth and adoption of agile as a practical means of delivery. Their direct interactions with agile delivery teams create a unique opportunity to expand their influence in generating valuable outcomes for end-users, consumers, customers, employees or suppliers. To do so, they need to further extend themselves— both in terms of skills and engagement—to remain relevant in today’s complex delivery environment.
How do you feel the scrum master role has evolved? Are newly minted project managers the scrum masters of tomorrow?
Great preparation goes into identifying the right project manager for the job—including determining the project’s delivery complexity, defining the role profile, selecting interview questions and validating professional certifications.
However, the interview process shouldn’t end once the new project manager is hired. A recurring interview process ensures project managers remain a good fit. It also helps showcase a project manager’s capabilities to instill confidence among leadership groups, stakeholders and team members—especially if elements drastically change, as they are wont to do.
Not every project manager is a good fit for every project. Original assumptions that lead to the initial acquisition of a project manager may not hold true as the project progresses. And poor outcomes often result from hasty decisions to get a project manager on-boarded as quickly as possible to start a project within a desired timeframe.
Here are three questions that not only ascertain the health of a project, but also the fit of the project manager. Depending on the outcome, you may choose to retain the project manager or replace them with someone who is a better fit.
1. Where are we now?
Being able to confidently articulate and identify the true position of a project and the recent progress velocity to get to that position is a foundation of project management success. Failure to know where the project currently resides puts future progress at risk.
Assisting the project manager in this determination of project position includes schedule and budget performance metrics, resource availability, dependencies, risk, issues and other inorganic position indicators. In addition, a project manager should be able to organically identify the “so what” implications and potential remedies required to create a three-dimensional view of project progress.
2. Where will we be in six weeks?
An old adage says that a point shows a current position, two points make a line and three points make a trend. Project managers should be constantly triangulating their project trajectory from their current position. If they can’t, they’re putting the project’s finish in jeopardy.
This six-week timeframe means a project manager can have a clear vision of the visible road ahead, but isn’t so far where they have to speculate well beyond a reasonable horizon.
Use of predictive quantitative methods and tools and prior project experience can help a project manager confidently state where the project is headed.
3. What changed from the original project scope?
Change is constant. It takes many forms and has diverse impacts. Additions or revisions of functional requirements, technical requirements, different implementation approaches, new expectations, supplier complexity, unfunded mandates and other events make up the aggregate, ever-changing landscape of a project.
While the project manager does his or her best to control identification, processing and action around changes, in some cases the aggregate impact of change can overwhelm.
In many cases, changes—such as leadership changes, new suppliers, as well as portfolio management actions that can merge existing projects—have nothing to do with the project manager’s capability. But when the depth and breadth of project change exceeds the capability of the project manager, it may be time to secure a replacement.
What line of questioning might you use to ensure that a project manager continues to be a good fit for the project they were hired for?
In my last post A Better Path Forward For Federal Programs , I discussed how the Program Management Improvement Accountability Act empowers the Office of Management and Budget to create a program and project management strategy for the U.S. federal government.
The legislation also requires the heads of several U.S. government agencies—including the Departments of Agriculture, Department of Labor, Department of Commerce, Department of Energy and Department of Education—to designate one senior team member to serve as its program management improvement officer. In this role, the senior team member will be responsible for implementing program management policies established by the agency and developing a strategy for enhancing the role of program managers within the agency.
The program management improvement officer also has another set of responsibilities that I find particularly interesting. The law says the project management improvement officer must develop a strategy for enhancing the role of program managers within the agency. This includes expanding training and educational opportunities for program managers. This portion of the legislation creates a formal process for program managers to strengthen their existing competencies and allows project managers to develop into program managers (I once wrote a post on this topic).
Given the complexities inherent to contemporary program management, professional development initiatives will successfully prepare program managers for progressively larger delivery responsibilities. In addition, they will create an opportunity to centralize lessons learned on existing delivery programs for even more effective future program management.
Admittedly, when I first heard of this legislation, I was somewhat doubtful of its ability to influence program management results. However, after diving into the details, I’ve become an advocate. I’m excited about the new standard it will set for federal program delivery—and the prospects it holds for building similar program management capabilities in the private sector.
We may jest about the effectiveness of government regulations, policies and practices—but this legislation has the potential to significantly boost program management innovation in the public sector.
Do you believe the Program Management Improvement Accountability Act will spur program delivery improvements in your workplace?