Project Management

Disciplined Agile

by , , , , , ,
#ChooseYourWoW | #ContinuousImprovement | #Kaizen | #ProcessImprovement | Adoption | agile | Agile certification | agile transformation | Analogy | Architecture | architecture | book | Business Agility | Certification | Choose your WoW | CMMI | Coaching | Collaboration | Compliancy | Configuration management | Construction phase | Context | Continuous Improvement | COVID-19 | Culture | culture | DAD | DAD discussions | DAD roles | Data Management | database | DevOps | Discipline | disciplined agile delivery | Documentation | DW/BI | Enterprise Agile | Enterprise Architecture | Enterprise Awareness | Essence | Evolving DA | Experiment | Financial | GDD | Geographic Distribution | global development | Goal-Driven | goal-driven | goals | Governance | Guideline | Hybrid | Improvement | inception | Inception phase | Kanban | Large Teams | Lean | Lifecycle | lifecycle | Metrics | mindset | News | News and events | Non-Functional Requirements | non-functional requirements | Operations | Outsourcing | People | Philosophies | Planning | PMI | PMI and DA | Portfolio Management | Practices | Principle | Process | process improvement | Product Management | Product Owners | Program Management | Project Management | Promise | quality | Release Management | Requirements | requirements | Reuse Engineering | Risk management | RUP | Scaling | scaling | scaling agile | Scrum | Support | Surveys | Teams | Technical Debt | Terminology | Testing | testing | Toolkit | Transformation | Workflow | show all posts

About this Blog

RSS

View Posts By:

Scott Ambler
Glen Little
Mark Lines
Valentin Mocanu
Daniel Gagnon
Michael Richardson
Joshua Barnes

Recent Posts

Failure Bow: Choosing Between Life Cycles Flowchart Update

Evolving Disciplined Agile: Guidelines of the DA Mindset

Evolving Disciplined Agile: Promises of the DA Mindset

Evolving Disciplined Agile: Principles of the DA Mindset

Evolving Disciplined Agile: The DA Mindset

Apply Consistent Metrics Categories Across an Agile Portfolio

Metrics A common question that we get from customers who are new to Disciplined Agile (DA) is how do you roll up metrics from solution delivery teams into a portfolio dashboard?  A more interesting question is how do you do this when the teams are working in different ways?  Remember, DA teams choose their way of working (WoW) because Context Counts and Choice is Good.  Even more interesting is the question “How do you roll up team metrics when you still have some traditional teams as well as some agile/lean teams?”  In this blog we answer these questions one at a time, in order.

Note: We’re going to talk in terms of a single portfolio in this article, but the strategies we describe can apply at the program (a large team of teams) too, and then the program-level metrics are further rolled up to higher levels.

 

How Do You Roll Up Agile Team Metrics Into a Portfolio Dashboard?

Pretty much the same way you roll up metrics from traditional teams.  There tends to be several potential challenges to doing this, challenges which non-agile teams also face:

  1. You can only roll-up metrics with similar units. You do need to be careful with some measures, such as team velocity, because the units vary across teams.  It is possible to enforce a common way to measure velocity across teams, but this tends to be more effort than it’s worth in practice. Sometimes there are metrics which you think you should be able to roll up but you discover (hopefully) that you really shouldn’t.  One example of this is number of defects by severity.  You can roll this metric up when the severity of a defect is determined in a consistent manner across teams, but this isn’t always the case in practice.
  2. Sometimes the math gets a bit complex.  Rolling up metrics isn’t always based on simple addition.  Many times you will need to weight metrics based in terms of time, size, financial impact or combinations thereof.  Interestingly, although some metrics can’t be rolled up because they are measured in different units, you can often roll up the trends of those metrics.  For example, acceleration is the change in velocity of a team.  Given an appropriate weighting formula you can roll up an average acceleration figure across teams.
  3. Some people believe you can only roll up the same metric. Basically, when a common metric is captured across teams (for example Net Promoter Score (NPS) or cyclomatic complexity) then you can easily (albeit with some “complex” math in some cases) roll them up to a program or portfolio level.  In the Govern Delivery Team process goal we refer to this strategy as consistent metrics, an option of the Provide Transparency decision point. This strategy works well when teams actually collect the same metrics, but in when teams choose their WoW this isn’t always the case.

 

How Do You Roll Up Agile Team Metrics Into a Portfolio Dashboard When the Teams Choose Their WoW, and it’s Different For Each Team?

When a team is allowed to choose it’s way of working (WoW), or “own their own process,” the team will often choose to measure itself in a manner that is appropriate to it’s WoW. This makes a lot of sense because to improve your WoW you will want to experiment with techniques, measure their effectiveness for your team within your current context, and then adopt the techniques that work best for you.  So teams will need to have metrics in place that provide them with insight into how well they are working, and because each team is unique the set of metrics they collect will vary by team.  For example, in Figure 1 below we see that the Data Warehouse (DW) team has decided to collect a different sent of metrics to measure stakeholder satisfaction than the Mobile Development team.  The DW team needs to determine which reports are being run by their end users, and more importantly they need to identify new reports that provide valuable information to end users – this is why they have measures for Reports run (to measure usage) and NPS (to measure satisfaction).  The Mobile team on the other hand needs to attract and retain users, so they measure things like session length and time in app to determine usage, and user retention and NPS to measure satisfaction.

Figure 1. Applying consistent metrics categories across disparate teams (click on it for a larger version).

Agile metrics categories

Furthermore, the nature of the problem that a team faces will also motivate them to choose metrics that are appropriate for them. In Figure 1 we see that each team has a different set of quality metrics: the DW team measures data quality, the mobile team measures code quality, and the package implementation team measures user acceptance test (UAT) results. Although production incidents and automated test coverage are measured by all three teams, the remaining metrics are unique.

The point is that instead of following the consistent metrics practice across teams by insisting that each team collects the same collection of metrics, it is better to ask for consistent metric categories across teams. So instead of saying “thou shalt collect metrics X, Y, and Z” we instead say “Thou shalt collect metrics that explore Category A, Category B, and Category C.” So, as you can see in Figure 1, each team is asked to collect quality metrics, time to market metrics, and stakeholder satisfaction metrics but it is left up to them what metrics they will choose to collect. The important point is that they need to collect sufficient metrics in each category to provide insight into how well the team addresses it. This enables the teams to be flexible in their approach and collect metrics that are meaningful for them, while providing the governance people within our organization the information that they need to guide the teams effectively.

So how do you roll up the metrics when they’re not consistent across teams?  Each team is responsible for taking the metrics that they collect in each category and calculating a score for that category.  It is likely that a team will need to work with the governance body to develop this calculation.  For example, in Figure 2 we see that the each team has a unique dashboard for their team metrics, yet at the portfolio level the metrics are rolled up into a stoplight status scorecard strategy for each category (Green = Good, Yellow = Questionable, Red = Problem). Calculating a stoplight value is one approach, you could get more sophisticated and calculate a numerical score if you like.  This is something the governance body would need to decide upon and then work with teams to implement.

Figure 2. Rolling up metrics categories (click on it for a larger version).

Agile metrics - portfolio dashboard

 

From the looks of the Portfolio dashboard in Figure 2 there is a heat map indicating the overall status of the team (using green, yellow, and red again) and the size of the effort (indicated by the size of the circle). Anyone looking at the portfolio dashboard should be able to click on one of the circles or team stoplights and be taken to the dashboard for that specific team. The status value for the heatmap would be calculated consistently for each team based on the category statuses for that team – this is a calculation that the governance body would need to develop and then implement.  The size of the effort would likely come from a financial reporting system or perhaps your people management systems.

 

How Do You Roll Up Team Metrics When Some Teams Are Still Traditional?

With a consistent categories approach it doesn’t really matter what paradigm the team is following.  You simply allow them to collect whatever metrics are appropriate for their situation within each category and require them to develop the calculation to roll the metrics up accordingly.  If they can’t come up with a reasonable calculation then the worst case would be for the Team Lead (or Project Manager in the case of a traditional team) to manually indicate/enter the status value for each category.

 

Parting Thoughts

For the consistent categories strategy to work the governance people need to be able to look at the dashboard for a team, which will have a unique collection of widgets on it, and be able to understand what the dashboard indicates. This will require some knowledge and sophistication from our governance people, which isn’t unreasonable to ask for in our opinion. Effective leaders know that metrics only provide insight but that they shouldn’t manage by the numbers. Instead they should follow the lean concept of “gemba” and go see what is happening in the team, collaborating with them to help the team understand and overcome any challenges they may face.

 

Related Links

Posted by Scott Ambler on: September 04, 2018 12:19 PM | Permalink | Comments (0)

Strategies for Tracking Time on Agile Teams

Time Tracking

In Time Tracking and Agile Software Development we overviewed why teams should consider tracking their time.  Primary reasons include:

  • You’re billing your customer by the hour
  • Your organization wants to account for CapEx/OpEx
  • Your organization wants to take advantage of tax credits (typically for R&D work)

A secondary reason to track time is because the team wants to measure where they are spending time so as to target potential areas to improve.  This is more of a side benefit than anything else – if this was your only reason to track time you’d be better off simply discussing these sorts of issues in your retrospectives.  But if you were already tracking time then running a quick report to provide the team with intel likely makes sense for you.

So what are your options for recording time?  Potential strategies, which are compared in the following table, include:

  1. Automated report from an agile management tool. The basic idea is to extract data from an agile management tool (JIRA, TFS, LeanKit, …) and load it into your time tracking system.
  2. Manual input by team members. Each team member, typically once a week, inputs their time into the time tracking tool.
  3. Manual input by the Team Lead. The Team Lead (ScrumMaster) inputs the time for their team into the time tracking tool.
  4. Manual input by a Project Manager/Coordinator. A PM or Project Coordinator, often in a support role to the team, inputs the time of team.
  5. Don’t track time at all. ‘Nuff said.

Table: Comparing options for tracking time.

Strategy Advantages Disadvantages
Automated report from agile management tool
  • Very efficient because it doesn’t require ongoing data input
  • Sufficient for CapEx/OpEx purposes
  • Sufficient for customer billing when the billing units are by the day (or greater)
  • Requires a bit of development work to feed data from your agile management tool into your time tracking system
  • May motivate the team to start treating the agile management tool like a time tracking tool (which often negates the value of the management tool)
  • Often requires a bit of (programmatic) fudging of the data to calculate the time not captured in the tool (such as coordination meetings, demos, retrospectives, …)
  • May require a bit of negotiation with your organization’s auditors (if any)
  • Only an option for teams using agile management tools
  • Works well for teams that are working in a fairly consistent manner (i.e. mature teams that have gelled)
Manual input by team members
  • Potentially the most accurate approach
  • Sufficient for CapEx/OpEx, tax credits, and customer billing
  • Team members often perceive this as an overhead
  • People will be motivated to input what they believe management wants, particularly if any sort of rewards or punishments are thought to be connected
  • Potential for significant expense across the organization (a few minutes per person per week starts to add up) if this gets too detailed or complicated
  • For people working on multiple teams (a question idea anyway) time tracking often becomes onerous
Manual input by Team Lead
  • Shifts the data input burden away from the team
  • Sufficient for CapEx/OpEx and tax credits
  • Likely sufficient for customer billing
  • Not as accurate as other strategies
  • Takes the Team Lead away from leadership tasks
  • Requires the Team Lead to know what is going on within the team (which frankly should be a given)
Manual input by Project Manager/Coordinator
  • Same as manual input by Team Lead
  • Not as accurate as other strategies
  • Likely requires the PM to interview/badger team members to find out what they did during the week
  • Little better than “make work” for the PM
Don’t track time at all
  • No overhead for the team
  • Your organization may be losing out on tax credits

This blog posting was motivated by a conversation that I had with Stacey Vetzal on Twitter.

Related Reading

Posted by Scott Ambler on: May 29, 2017 06:57 AM | Permalink | Comments (0)

Should You Govern Agile Teams Via a Traditional Strategy?

Categories: Governance, Surveys

The quick answer is no, that’s an incredibly bad idea.

We ran a study in February 2017, the 2017 Agile Governance survey, to explore the issues around governance of agile teams. This study found that the majority of agile teams were in fact being governed in some way, that some agile teams were being governed in an agile or lightweight manner and some agile teams in a traditional manner.  See the blog Are Agile Teams Being Governed? for a summary of those results.

The study also examined the effect of governance on agile teams, exploring the perceived effect of the organization’s governance strategy on team productivity, on the quality delivered, on IT investment, and on team morale.  It also explored how heavy the governance strategy was perceived to be and how well it was focused on the delivery of business value. The following figure summarizes the results of these questions.

Governance Effectiveness with Agile Teams

Here are our conclusions given these results:

  1. Agile governance helps agile teams. There is a clear co-relation between an agile approach to governing agile teams and positive results such as improving productivity, increasing quality, spending your investment in IT wisely, and improved team morale. This is what we believe the goal to be, to help the people being governed to be more effective and successful.
  2. Traditional governance hinders agile teams.  There is a clear co-relation between traditional approaches to governing agile teams and reduced team productivity, reduced quality of output, wasting IT investment, and decreased team morale.  We believe that these results are the exact opposite of what you hope to achieve with your governance strategy.
  3. Agile teams should be governed in an agile manner.  This follows directly from the previous two conclusions.  It should come as no surprise that your governance strategy should be well-aligned with what it is being governed.
  4. Traditional governance strategies likely hinders traditional teams too.  We didn’t look into this issue directly, but our experience has been that traditional governance tends to be more of a hindrance than a help to traditional teams as well.

When we work with organizations to help them to adopt agile ways of working, we often find that they are running into several common challenges when it comes to IT governance:

  1. They have both agile teams and traditional software teams.  This is because it’s a multi-modal world: You will have some teams taking a traditional approach, some an agile approach, some take a lean approach, and some are even skilled enough for continuous delivery.  Each team will follow the lifecycle that makes the most sense for them, and as a result each team should be governed by the approach that best suits the way that they are working.  To do anything different would be to hinder the teams, and that isn’t what good governance should be about.
  2. There is a desire for a single approach to governing software teams. This makes sense on the surface because it would simplify your overall governance strategy, thereby making things easier for the people doing the governing.  But, as we’ve learned, this results in negative effects in practice.  Your governance strategy must be flexible enough to support the range of teams being governed.
  3. The governance team is struggling to understand agile.  Your executives and middle management need education and coaching in agile and lean just like the people on your software team do.  It is naive to expect your governance people to devise a governance strategy for agile when they don’t really understand the implications of agile to begin with.

For agile to succeed in your organization the way that you approach IT must evolve to enable and support it, and this includes your governance strategy.  Reach out to us if you would like some help in addressing this important issue.

Related Reading

 

Posted by Scott Ambler on: April 08, 2017 07:11 AM | Permalink | Comments (0)

Are Agile Teams Being Governed?

Categories: Governance, Surveys

For the major of teams the answer is yes.  We ran a survey in February 2017, the 2017 Agile Governance survey, to explore the issues around governance of agile teams.  As you can see in the following diagram, 78% of respondents indicated that yes, their agile teams were in fact being governed in some manner.

Agile governance rates

We also asked people about the approach to governing agile teams that their organization followed.  As you can see in the following diagram, a bit more than a third of respondents indicated that the governance strategy was lightweight or agile in nature.  Roughly the same indicated that their agile teams had a more traditional approach to governance applied to them, and one quarter said their governance approach was neither helping nor hindering their teams.

How are agile teams being governed?

Governance tends to be a swear word for many agilists and they will tell you that governance is nothing than useless bureaucracy.  Sadly in many organizations this seems to be the case.    In the next blog in this series we will compare the effectiveness of agile and traditional strategies for governing agile teams.

Related Reading

Posted by Scott Ambler on: April 04, 2017 07:32 AM | Permalink | Comments (0)

Acceleration: An Agile Productivity Metric

Metrics

A common question that customers ask us is how do you measure productivity on agile teams. If you can easily measure productivity you can easily identify what is working for you in given situations, or what is not working for you, and adjust accordingly. One way to do so is to look at acceleration, which is the change in velocity.

A common metric captured by agile teams is their velocity. Velocity is an agile measure of how much work a team can do during a given iteration. Velocity is typically measured using an arbitrary point system that is unique to a given team or program. For example, my team might estimate that a given work item is two points worth of effort whereas your team might think that it’s seven points of effort, the important thing is that it’s consistent. So if there is another work item requiring similar effort, my team should estimate that it’s two points and your team seven points. With a consistent point system in place, each team can accurately estimate the amount of work that they can do in the current iteration by assuming that they can achieve the same amount of work as last iteration (an XP concept called “yesterday’s weather”). So, if my team delivered 27 points of functionality last iteration we would reasonably assume that all things being equal we can do the same this iteration.

It generally isn’t possible to use velocity as a measure of productivity. You can’t compare the velocity of the two teams because they’re measuring in different units. For example, we have two teams, A and B, each of 5 people and each working on a web site and each having two-week long iterations. Team A reports a velocity of 17 points for their current iteration and team B a velocity of 51 points. They’re both comprised of 5 people, therefore team B must be three times (51/17) as productive as team A. No! Team A is reporting in their point system and B in their point system, so you can’t compare them directly. The traditional strategy, one that is also suggested in the Scaled Agile Framework (SAFe), would be to ask the teams to use the same unit of points.  This might be a viable strategy with a small number of teams although with five or more teams it would likely require more effort than it was worth.  Regardless of the number of teams that you have it would minimally require some coordination to normalize the units and perhaps even some training and development and support of velocity calculation guidelines. Sounds like unnecessary bureaucracy that I would prefer to avoid. Worse yet, so-called “consistent” measurements such as FPs are anything but consistent because there’s always some sort of fudge factor involved in the calculation process that will vary by individual estimator.

An easier solution exists. Instead of comparing velocities you instead calculate the acceleration of each team. Acceleration is the change in velocity over time. The exact formula for acceleration is:

(NewVelocity – InitialVelocity)/InitialVelocity

For example, consider the reported velocities of each team shown in the table below. Team A’s velocity is increasing over time whereas team B’s velocity is trending downwards. All things being equal, you can assume that team A’s productivity is increasing whereas B’s is decreasing. At the end of iteration 10, if we wanted to calculate the acceleration since the previous iteration (#9), it would be (23-22)/22= 4.5% for Team A and (40-41)/41 = -2.4% for Team B.  So Team A improved their productivity 4.5% during iteration 10 and Team decreased their productivity 2.4% that iteration.  A better way to calculate acceleration is to look at the difference in velocity between multiple iterations as this will help to smooth out the numbers over time because as you see in the table the velocity will fluctuate naturally over time (something scientists might refer to as noise).  Let’s calculate acceleration over 5 iterations instead of just one, in this case comparing the differences in velocity from iteration 6 to iteration 10.  For Team A the acceleration would be (23-20)/20 = 15% and for Team B (40-45)/45 = -11.1% during the 5 iteration period, or 3% and -3.7% respectively on average per iteration.

Iteration Team A Velocity Team B

Velocity

1 17 51
2 18 49
3 17 50
4 18 47
5 19 48
6 20 45
7 19 44
8 21 44
9 22 41
10 23 40

 

Normalizing Acceleration for Team Size

The calculations that we performed above assumed that everything on the two teams remained the same. That assumption is likely a bit naïve.  It could very well be that people joined or left either of those teams, something that would clearly impact the team’s velocity and therefore it’s acceleration.  Let’s work through an example.  We’ve expanded the first table to include the size of the team each iteration.  We’ve also added a column showing the average velocity per person per iteration for each team, calculated by dividing the velocity by the team size for that iteration.  Taking the effect of team size into account, the average acceleration between the last five iterations for Team A is (1.9-1.8)/1.8/5 = 1.1% and for Team B is (5-5)/5/5 = 0.

Iteration Team A Velocity Team A Size Team A Velocity Per Person Team B

Velocity

Team B Size Team B Velocity Per Person
1 17 10 1.7 51 10 5.1
2 18 10 1.8 49 10 4.9
3 17 11 1.5 50 10 5
4 18 11 1.6 47 9 5.2
5 19 11 1.7 48 9 5.3
6 20 11 1.8 45 9 5
7 19 12 1.6 44 8 5.5
8 21 12 1.8 44 8 5.5
9 22 12 1.8 41 8 5.1
10 23 12 1.9 40 8 5

Similarly, perhaps there was a holiday during one iteration. When there are ten working days per iteration and you lose one or more of them due to holidays it can have a substantial impact on velocity as well.  As a result you may want to take into account the number of working days each iteration in your calculation.  You would effectively calculate average acceleration per person per day in this case.  Frankly I’m not too worried about that issue as it would affect everyone within your organization in pretty much the same way, and it’s easy to understand why there was a “blip” in the data for that iteration.

 

What Does Acceleration Tell You?

For how you use acceleration in practice, there are three scenarios to consider:

  1. Positive acceleration. This is an indication that productivity may be rising on the team, although it does not indicate the cause of that increase.
  2. Zero acceleration. This is an indication that the team’s productivity is remaining flat, and that perhaps they should consider doing retrospectives regularly and then act on the results from those retrospectives. Better yet they can “dial up” their process improvement efforts by adopting something along the lines of the Disciplined Agile Framework.
  3. Negative acceleration. If the acceleration is negative then productivity on the team is going down, likely an indicator of quality and/or team work problems.

Of course it’s not wise to govern simply by the numbers, so instead of assuming what is going on we would rather go and talk with the people on the two teams. Doing so you might find out that team A has adopted quality-oriented practices such as continuous integration and static code analysis which team B has not, indicating that you might want to help team A share their learnings with other teams.

 

Monetizing Acceleration

This is fairly straightforward to do. For example, assume your acceleration is 0.7%, that there are five people on the team, your annual burdened cost per person is $150,000 (your senior management staff should be able to tell you what this number is), and that you have two week iterations. So, per iteration the average burdened cost per person must be $150,000/26 = $5,770. Productivity improvement per iteration for this team must be $5,770 * 5 * .007 = $202. If the acceleration stayed constant at 0.7% the overall productivity improvement for the year would be (1.007)^26 (assuming the team works all 52 weeks of the year) which would be 1.198 or 19.8%. This would be a savings of $148,500 (pretty much the equivalent of one new person).

Another approach is calculate the acceleration for the year by comparing the velocity from the beginning of the year to the end of the year (note that you want to avoid comparing iterations near any major holidays). So, if the team velocity the first week of February 2015 was 20 points, the same team’s velocity the first week of February 2016 was 23 points, that’s an acceleration of (23-20)/20 = 15% over a one year period, for a savings of $112,500.

 

Advantages of the Acceleration Metric

There are several advantages to using acceleration as an indicator of productivity over traditional techniques such as FP counting:

  1. It’s easy to calculate. We worked through two common variations earlier, you’ll need to experiment to determine what works for you.
  2. It is inexpensive. Acceleration is based on information already being collected by the team, their velocity, so there is no extra work to be done by the team.  Assuming that the team hasn’t decided to take a #NoEstimates approach
  3. It is easy to automate. For example, most agile management tools (e.g. VersionOne, Rally, Jira, Microsoft TFS) calculate velocity automatically from their work item list/product backlog and do velocity trend reporting via their team dashboard functionality. This trend reporting is effectively a visual representation of the team’s acceleration (or deceleration as the case may be).
  4. You can easily adjust for changing team size.
  5. You can easily monetize this metric.
  6. It is unitless. The “units” are % change in points per iteration, or % change in points per time period depending on the way that you want to look at it. Because it’s a percentage you can use it as a basis of comparison.
  7. You apply this across a department. It is fairly straightforward to roll up the acceleration of project teams into an overall acceleration measure for a larger program or portfolio simply by taking a weighted average based on team size. However, this is only applicable to teams that are in a position to report an accurate acceleration (the agile and iterative teams) and of course are willing to do so.

 

Potential Disadvantages of Acceleration

Of course, nothing is perfect, and there are a few potential disadvantages:

  1. It can be gamed. Acceleration is derived from velocity which in turn is derived from manually-collected measures, and anything gathered manually can be easily gamed.
  2. Management must be flexible. For this to be acceptable senior management must be willing to think outside the “traditional metrics box”. Using a non-standard, simple metric to calculate productivity? Preposterous! Directly measuring what you’re truly interested in instead of calculating trends over long periods of time? Doubly preposterous!
  3. The terminology sounds scientific. Terms such as velocity and acceleration can motivate some of us to start believing that we understand the “laws of IT physics”, something which we doubt very highly that as an industry we understand.

We hope that you’ve found this blog post valuable.

Posted by Scott Ambler on: June 13, 2016 10:13 AM | Permalink | Comments (0)
ADVERTISEMENTS

Love can sweep you off your feet and carry you along in a way you've never known before. But the ride always ends, and you end up feeling lonely and bitter. Wait. It's not love I'm describing. I'm thinking of a monorail.

- Jack Handey

ADVERTISEMENT

Sponsors