A common question that we get from customers who are new to Disciplined Agile (DA) is how do you roll up metrics from solution delivery teams into a portfolio dashboard? A more interesting question is how do you do this when the teams are working in different ways? Remember, DA teams choose their way of working (WoW) because Context Counts and Choice is Good. Even more interesting is the question “How do you roll up team metrics when you still have some traditional teams as well as some agile/lean teams?” In this blog we answer these questions one at a time, in order.
Note: We’re going to talk in terms of a single portfolio in this article, but the strategies we describe can apply at the program (a large team of teams) too, and then the program-level metrics are further rolled up to higher levels.
How Do You Roll Up Agile Team Metrics Into a Portfolio Dashboard?
Pretty much the same way you roll up metrics from traditional teams. There tends to be several potential challenges to doing this, challenges which non-agile teams also face:
How Do You Roll Up Agile Team Metrics Into a Portfolio Dashboard When the Teams Choose Their WoW, and it’s Different For Each Team?
When a team is allowed to choose it’s way of working (WoW), or “own their own process,” the team will often choose to measure itself in a manner that is appropriate to it’s WoW. This makes a lot of sense because to improve your WoW you will want to experiment with techniques, measure their effectiveness for your team within your current context, and then adopt the techniques that work best for you. So teams will need to have metrics in place that provide them with insight into how well they are working, and because each team is unique the set of metrics they collect will vary by team. For example, in Figure 1 below we see that the Data Warehouse (DW) team has decided to collect a different sent of metrics to measure stakeholder satisfaction than the Mobile Development team. The DW team needs to determine which reports are being run by their end users, and more importantly they need to identify new reports that provide valuable information to end users – this is why they have measures for Reports run (to measure usage) and NPS (to measure satisfaction). The Mobile team on the other hand needs to attract and retain users, so they measure things like session length and time in app to determine usage, and user retention and NPS to measure satisfaction.
Figure 1. Applying consistent metrics categories across disparate teams (click on it for a larger version).
Furthermore, the nature of the problem that a team faces will also motivate them to choose metrics that are appropriate for them. In Figure 1 we see that each team has a different set of quality metrics: the DW team measures data quality, the mobile team measures code quality, and the package implementation team measures user acceptance test (UAT) results. Although production incidents and automated test coverage are measured by all three teams, the remaining metrics are unique.
The point is that instead of following the consistent metrics practice across teams by insisting that each team collects the same collection of metrics, it is better to ask for consistent metric categories across teams. So instead of saying “thou shalt collect metrics X, Y, and Z” we instead say “Thou shalt collect metrics that explore Category A, Category B, and Category C.” So, as you can see in Figure 1, each team is asked to collect quality metrics, time to market metrics, and stakeholder satisfaction metrics but it is left up to them what metrics they will choose to collect. The important point is that they need to collect sufficient metrics in each category to provide insight into how well the team addresses it. This enables the teams to be flexible in their approach and collect metrics that are meaningful for them, while providing the governance people within our organization the information that they need to guide the teams effectively.
So how do you roll up the metrics when they’re not consistent across teams? Each team is responsible for taking the metrics that they collect in each category and calculating a score for that category. It is likely that a team will need to work with the governance body to develop this calculation. For example, in Figure 2 we see that the each team has a unique dashboard for their team metrics, yet at the portfolio level the metrics are rolled up into a stoplight status scorecard strategy for each category (Green = Good, Yellow = Questionable, Red = Problem). Calculating a stoplight value is one approach, you could get more sophisticated and calculate a numerical score if you like. This is something the governance body would need to decide upon and then work with teams to implement.
Figure 2. Rolling up metrics categories (click on it for a larger version).
From the looks of the Portfolio dashboard in Figure 2 there is a heat map indicating the overall status of the team (using green, yellow, and red again) and the size of the effort (indicated by the size of the circle). Anyone looking at the portfolio dashboard should be able to click on one of the circles or team stoplights and be taken to the dashboard for that specific team. The status value for the heatmap would be calculated consistently for each team based on the category statuses for that team – this is a calculation that the governance body would need to develop and then implement. The size of the effort would likely come from a financial reporting system or perhaps your people management systems.
How Do You Roll Up Team Metrics When Some Teams Are Still Traditional?
With a consistent categories approach it doesn’t really matter what paradigm the team is following. You simply allow them to collect whatever metrics are appropriate for their situation within each category and require them to develop the calculation to roll the metrics up accordingly. If they can’t come up with a reasonable calculation then the worst case would be for the Team Lead (or Project Manager in the case of a traditional team) to manually indicate/enter the status value for each category.
For the consistent categories strategy to work the governance people need to be able to look at the dashboard for a team, which will have a unique collection of widgets on it, and be able to understand what the dashboard indicates. This will require some knowledge and sophistication from our governance people, which isn’t unreasonable to ask for in our opinion. Effective leaders know that metrics only provide insight but that they shouldn’t manage by the numbers. Instead they should follow the lean concept of “gemba” and go see what is happening in the team, collaborating with them to help the team understand and overcome any challenges they may face.
IT Portfolio Management addresses how an IT organization goes about identifying, prioritizing, organizing, and governing their various IT endeavors. Disciplined Agile Portfolio Management seeks to do this in a lightweight and streamlined manner that maximizes the creation of business value in a long-term sustainable manner. IT endeavors typically include solution delivery initiatives/projects, stable product development teams, business experiments (along the lines of a lean startup strategy), and the operation of existing IT-based solutions.
Being agile, having an agile mindset, is foundational to working in an agile manner. The Disciplined Agile Manifesto and the principles of lean software development provide an important start at this mindset. In this blog we explore similar agile philosophies that are specific to successful portfolio management. These philosophies are:
Our experience is that the philosophies describe above enable portfolio managers to be more effective in practice. We hope you have found this blog of value and we welcome your feedback.
The basic idea with rolling wave planning is that you plan things that are near in time to you in detail and things that are distant in time at a higher level. The thinking is that the longer away in time that something is the greater the chance that it will change during that time, therefore any investment in thinking through the details is likely wasted. You still want to plan at a high level to both guide your current decisions and to set people’s expectations as to what is likely to come.
Rolling wave planning is implemented in several places of the DA toolkit. First, as you can see in Figure 1 below, it is an option of the Level of Detail decision point of the Develop Initial Release Plan process goal. A rolling wave approach to release planning has the advantages of more accurate and flexible planning although can be a bit disconcerting to traditional managers who are used to annual planning strategies.
Figure 1. The Develop Initial Release Plan goal diagram.
The Portfolio Management process blade supports rolling wave budgeting as an option for its Manage the Budget decision point. This is depicted in Figure 2. The advantages are greater flexibility and greater likelihood of investing your IT funding more effectively, albeit at the loss of the false predictability provided by an annual budgeting strategy.
Figure 2. The goal diagram for the Portfolio Management process blade.
The Program Management process blade supports rolling wave planning of a program itself, as you seen in Figure 3. Planning and coordination are critical on a large program, and rolling wave planning offers the advantages greater flexibility, the ability to think important cross-team issues through, and the ability to react to changing stakeholder needs. The primary disadvantage is that it can be disconcerting for traditionalists who are used to thinking every thing through from the beginning.
Figure 3. The goal diagram for the Program Management process blade.
As you can see in Figure 4, rolling wave strategies can be applied in Product Management to evolve the business vision/roadmap. A continuous, rolling wave approach is critical to your success because the market place changes so quickly – these days, few organizations can tolerate an annual approach to business planning and in the case of companies with external customers an ad-hoc approach can prove to be too unpredictable for them.
Figure 4. The goal diagram for the Product Management process blade.
Previously we saw that rolling wave strategies can be applied to evolve your technology roadmap, as indicated in the goal diagram for Enterprise Architecture in Figure 5. The advantages of this approach are that your roadmap evolved in sync with both changes in technology and with your organization’s rate of experimentation and learning. The main disadvantage is that your technology roadmap is effectively a moving target.
Figure 5. The goal diagram for the Enterprise Architecture process blade.
As you can see, rolling wave strategies are an integral part of the Disciplined Agile (DA) toolkit. In fact, in most situations they prove to be the most effective and flexible strategies available to you. The advantages of rolling wave planning tend to greatly outweigh the disadvantages. More on this next time.
On February 23, 2016 I gave a webinar entitled (In Agile) Where do all the Managers go? A recording of the webinar is posted on Youtube and a PDF of the slides on Slideshare. This blog overviews the webinar and provides answers to the numerous questions that were asked during it.
The webinar began with a discussion of four trends that are reducing the need for people in management positions:
We then discussed the options that existing managers have in an agile environment. In Disciplined Agile there are four roles that existing managers are likely to transition to: Team Lead, Product Owner, Team Member, and Specialist. Specialist roles – such as Data Manager, Portfolio Manager, Program Manager, and Operations Manager – occur at scale and the corresponding positions are few and far between. Read the article Disciplined Agile Roles at Scale for more details.
We end with words of advice for existing managers: Observe what is actually happening; be flexible; and choose to evolve.
Questions and Answers
We’ve organized the questions into the following topics:
Evolving to New Roles
Will not the existing technical managers be disappointed with only people management work?
That depends on the person. Some will be very happy to do this, some will not.
How will managers fit into a leader role?
It depends on the person again. Some managers are very good leaders right now, some have the potential to be good leaders, and some don’t. They will need training and coaching to fit into their new role(s).
Addressing “Management Activities”
If there are no PMs in Agile, who handles communication with clients (meeting deadlines, priorities, etc.)
The Product Owner.
How does individual performance to be taken up in Agile team? I think that is more crucial and challenging for Agile Leader / Manager.
It is always difficult to address performance-related activities. There are many lines of thought on how to do this. The most progressive is for the Team Lead to provide feedback to team members on a just-in-time basis. If the Team Lead seems behaviour, either desirable or undesirable, but a team member then they should comment on it right away so as to reinforce or dissuade it as soon as possible. Many organizations still have an annual review process, a strategy that many organizations have abandoned due to it’s ineffectiveness, where functional managers get involved with the review process.
I have seen that you have selected the Team Lead as the responsible of assess team members and budgeting the project. In Scrum the Product Owner is compare to a CEO that’s the reason I would say the Product Owner is responsible for bugdeting and about assessing I prefer a more democratic form which involve all the members. So what do you think about PO managing the budget and a democratic assessing vs one single vision assess?
Yes, I misspoke during the webinar. The Product Owner is often responsible for the team’s budget and is responsible for reporting the current financial information to the stakeholders. The Team Lead is often responsible for similar reporting to their management team.
Having multiple people involved with reviews/feedback is usually a pretty good idea. The People Management process blade captures several potential strategies. However, it is still a good idea for the Team Lead to provide feedback as well, see my earlier answer.
Potential Management Roles
I think there is still a need a bridge manager role between Finance, Teams, and PMO type orgs to ensure Product owners have budget… views. Thoughts?
In smaller organizations this likely isn’t an issue. In larger organizations there is often a Portfolio Management effort that is responsible for such issues.
What might be potential responsibilities of an Operations Manager, Data Manager, …?
Please read the article Disciplined Agile Roles at Scale for descriptions of these roles.
Okay, the data management team needs a team manager/leader. Are large organizations using various resource managers? (Although would be less necessary with stable teams I would think)
Exactly. Large organizations still tend to have people in resource manager roles, although sometimes they have different titles such as CoE Lead or HR Manager, but with stable teams they need far fewer of them.
If the team has a Team Lead/Scrum Master that is only the servant leader for 1-2 teams, is it suitable to have people managers?
What value would a “people manager” bring to the team? This is the fundamental dilemma for managers, for everyone for that matter, when an organization moves to agile ways of working. If they’re not bringing real value to the team then they either need to find ways to do so, which likely isn’t whatever management activities they’re trying to cling to, or they need to go elsewhere and try to add value there.
Do you intend to update the DA 2.0 interative pic on the DAD site to talk about “Potential Management Roles at Scale” as mentioned in page 18 of this presentation?
Yes. We actually have something in beta that we haven’t released yet. We’re just about to release an update to the main picture, which in turn requires an update to the role version of the interactive pic.
What is the most basic difference between Project/Program/Portfolio Managers in Agile?
Quick answer is that there isn’t Project Managers in Disciplined Agile nor in methods such as Scrum, XP, and so on. At the program level (a large team of teams) you likely need someone in a Program Manager (or more accurately Program Coordinator) role to coordinate activities (see the Program Management process blade for details). A Portfolio Manager is focused on the IT level and should be concerned about pre-development activities, development/delivery teams that are currently in flight, as well as operational activities.
Also, please read the article Disciplined Agile Roles at Scale for descriptions of these roles.
How does one manage the career path of the Team Leads? Is there career progression beyond a TL to be a specialist or does s/he continue being a TL throughout his career?
Everyone is different, so there isn’t one exact answer. It depends on what the person wants to do and what positions are available to them. If their desire is to move into management then there are fewer IT management positions available to them. If they want to become an AO or PO then they need to work towards getting the skills and experience to fulfill those sorts of roles. The People Management process blade includes career management strategies.
How do you evaluate what roles are/will be necessary?
It depends on the needs of the team in the situation that they face. The primary delivery roles typically exist on all delivery teams and the secondary roles start to appear at scale.
How do you see the role of a BA in agile?
Most existing BAs, like most existing project managers, will need to transition to other roles. However, at scale there is a need for some people in the specialist BA role. I recently has a user group presentation recorded on this very topic. See Disciplined Agile Business Analysis: Lessons from the Trenches.
How do we approach a situation where management wants weekly status reports from a Program Manager who can combine both Team Lead & Product Owner roles, as well as manage multiple projects that may be similar in nature or not.
A few thoughts on this:
In a typical organization, where to team lead(s) report into?
It depends. We’ve seen them report into a Program Manager or a Portfolio Manager. During the transition effort a Project Management Office (PMO) may still exist so Team Leads might report into there, although we often find that there’s a serious cultural and mindset difference that can be very frustrating for everyone involved.
During Your Agile Transformation
What about managers being responsible to support an agile transformation journey in a large organization?
Yes, they would very likely be working as part of an Agile Center of Excellence (CoE), although that would be mostly staffed by experienced agile coaches. There is a need for one or more senior execs to sponsor your agile transformation.
How to deal with “Project Manager” role renamed as “Agile Project Manager” but expected to do the same responsibilities as traditional PM?
We see this sort of stuff all the time unfortunately. First thing to do is to get these people educated in how agile actually works in practice, we’d suggest DA 101: The Disciplined Agile Experience or DA 104: Introduction to Disciplined Agile as your best option to get the whole picture. Next, work through with them how they would actually add real value on the team (see the discussions earlier). Very likely many of the activities that they think need to be are being handled by someone else or have been automated away. Third, get them some coaching to help them to truly transition to agile.
On my project, I am the Team Lead and there is a Project Manager. So far, I have observed that there are several conflicts in responsibilities. How do we come to an agreement of who handles which responsibilities? For my next project, would you suggest I work on a project with no project manager?
We often have to run facilitated workshops in organizations where we work through the roles and responsibilities that are needed in practice. We do this with a wide range of people and we do so in a collaborative and public manner. You need to come to an agreement as to who does what. Doesn’t sound like that’s happened in your case. When you work through this sort of an exercise you quickly discover that you don’t need a project manager, although there may be some project control officer (PCO) responsibilities that would be assigned to either the team lead or some sort of administrative role (such as PCO).
Do you have any advice on how to deal with the removal of the traditional hierarchy – in a flattening of responsibilities, ‘reporting-lines’ and salaries (or having a vast range of skills and pay-scales all with a job title of ‘team member’?)
This is what an agile transformation will accomplish for your organization. It takes time and investment in your people to implement. I highly suggest that you get some experienced coaches to help you do this.
Management is severely, negatively, personally affected by Agile, and will not look fondly upon it in many cases. Any tips to reduce this? Do you recommend mass management reduction, or multiple smaller rounds?
The first step is to recognize that your organization doesn’t exist to create jobs for managers, regardless of what the managers may think. Agile is about focusing on value, so why wouldn’t a good manager be interested in being actively involved with doing so? My recommendation is always to get training and coaching for everyone, including managers. As I described in the webinar there are many options for existing managers in the agile world if they’re willing to be flexible and evolve. Your organization should choose to help people make these transitions to new roles. However, if people are not willing to make the transition then they shouldn’t be surprised if the find themselves being asked to seek employment elsewhere.
It sounds like the person asking about “who’s responsible for delivery” might have used “responsible” when they meant “accountable” – many managers are the single wringable neck for something in their job description. Do you feel Agile draws the same distinction between the two like ITSM, for instance, does?
Agile is based on a collaborative, teamwork-based mindset. Having said that, it does make sense to have someone ultimately responsible for certain things. For example, the Product Owner is responsible for prioritizing the work on an agile team. Similarly, you may have someone in the Release Manager role who is responsible for overall Release Management within your organization. This is particularly important for regulatory environments where by law you need to have someone not involved with development who makes the final decision as to whether the solution is released or not.
From your observations and experience, what is the average timeframe for the managers number to decrease? How long does the process of the shift take?
It depends. We’ve seen this happen over timeframes as short as six months to several years. With solid coaching this process will go a lot faster and smoother.
How to motivate and enable senior leaders to give up control?
In agile, particularly in Disciplined Agile, senior leaders have greater visibility and opportunities to steer than what they had in the traditional world. What they need to do is give up their false sense of control that traditional strategies provide. The real issue usually isn’t senior leaders but instead is middle management. They are the people who are currently performing many of the management tasks that are implemented in a more streamlined manner following agile approaches.
How can middle management start the agility journey when top leaders are not yet on board?
Agile typically begins following a stealth adoption strategy where senior leaders are unaware that it’s happening. The point is that anyone, including middle management, can start adopting agile strategies long before senior leadership gets involved. Strategies such as working collaboratively, enabling your team(s) to plan and organize their own work, adopting dashboard technology, and streamlining the bureaucracy whenever possible is very possible to accomplish on your own.
Thanks for being frank about the role(s) for managers in an evolving Agile culture: Agree, traditional project management organization’s aren’t highlighting these trends (and positive outcomes.)
You’re welcome. Traditional project management organizations often go at it from the point of view of how to continue justifying management activities. We go at it from the point of view of how to improve your overall organizational effectiveness and as a result come to a different conclusion.
Will the idea for stable team become stale after some years? People tend to get frustrated doing same work. What’s the solution in that case?
Stable teams evolve over time. You’ll get people joining the team every so often and similarly leaving the team every so often. It’s natural for people to want to move on and try something new every few years. As a result your organization will still need People Management activities in place that motivate and enable people to manage their careers.
As far as stable teams go, commonly Valve, Inc. is referred to a place where teams are formed around projects that the team members find the most interesting. Project leaders try to sell their project to get developers. Your thoughts?
This is great technique that other organizations may be able to adopt. Allowing teams to form themselves is likely the most effective way to do so. However, like all strategies, there are some potential disadvantages. Team culture may become ingrained and they will not attract people with a different culture who would have the potential to add some real value to the team otherwise.
Is there a method to build the stable teams? Domain, Product, line of business?
There are several strategies for doing this. The most common is to form feature teams that do all of the work to implement a feature as a vertical slice through your entire infrastructure. Another approach is to form component teams that work on a technical or domain component/framework/LoB. A third approach is internal open source. We’ve discussed these strategies in greater detail at Strategies for Organizing Large Agile Teams.
Do you think stable teams concept will work in service-based organisation?
Yes. It’s a bit more difficult because you’d be bringing entire customer projects to the team at once instead of a flow of smaller features. Of course you can break each large project up into smaller features and feed them to teams in an interleaved manner, requiring a sophisticated approach to requirements management.
Training and Certification
What baseline training do you recommend for agile managers?
A good place to start is training on agile thinking, often referred to as how to be agile. Then I would recommend training that describes the full delivery lifecycle from end-to-end, something like DA 101: The Disciplined Agile Experience or DA 104: Introduction to Disciplined Agile. You want to understand all aspects of the agile delivery process, not just the management ones. Scrum training is popular but far too narrow. SAFe training isn’t for beginners.
I would like to participate in a certification workshop/further training. There doesn’t seem to be many offerings in the US. Are there plans to expand training opportunities in the states?
Yes. In fact we have training coming up in the Baltimore area in March and Philadelphia in April. We will have more open enrollment workshops scheduled soon. Please visit the homepage of the Disciplined Agile Consortium for a listing of upcoming public workshops.
What should we be telling folks that have PMP’s – are they still valid? is PMP training moving toward Agile software development.
Yes, the PMI is moving towards agile but they have a very large ship to turn. Unfortunately the PMI training tends to suffer from the challenges that I described earlier – it seems to promote a rather unrealistic vision of how managers can potentially fit into agile.
In IT we are often asked to estimate the expected time/schedule or cost of software development. Sadly, the desire of stakeholders to have “predictable” schedules or costs results in significant dysfunction within a software development team. When a software team is forced by their stakeholders to commit to a schedule/cost they must then ensure that the schedule/cost doesn’t slip. For example, to protect themselves from increased time and cost due to scope creep, software development teams will make it difficult for stakeholders to change their requirements during Construction and even go so far as to drop promised scope late in a project. The desire of stakeholders to reduce their financial risk often results in behaviors by the software development team that ensure that stakeholders don’t get what they actually want. Naturally IT gets blamed for this.
We need to do better. In this blog we summarize the things that we know to be true about software development estimation. In no particular order, they are:
To summarize, when you are required to provide estimates for your software development efforts that you should take a pragmatic, light-weight approach to doing so. This blog posting has provided many practical insights that should help guide your decisions. These insights and many more, are built right into the Disciplined Agile (DA) toolkit.