Disciplined Agile

By Jeev Chugh, CDAP | CDAI and Joshua Barnes CDAC | CDAI

Enterprise agile transformation roadmaps using Lean Change are often fueled by the desire to deliver value faster with increased flexibility.  They start with replacing traditional (waterfall) delivery methods with an Agile way of working, using pilot teams to validate early decisions on how to experiment with aspects such as team formation, lifecycle (agile, lean, continuous delivery, etc.) practices and techniques, and so on.  What team members and business and technology leaders alike quickly glean is that the mind shift to collective ownership of the work, decentralized decision making, and all the energy to shift to self-organizing teams is just the first step.  A big step, but one in a long journey.

To truly achieve the sizable outcomes from these enterprise transformations, many aspects beyond that of individual agile teams achieving a good level of maturity in “their” agile ways of working are needed.  One of the shifts that enables such outcomes is adopting a Product Portfolio operating model.  In this model, we transition from assembling a team to deliver a fixed scope via a project and then disband as the project ends to a long-lasting stable team delivering business outcomes via continual improvement of a product.

Along with this change comes the realization that traditional waterfall methods of budgeting and project cost accounting that requires upfront plans and budgets, is in inherent conflict with Agile ways of working; especially so when the agile approach uses a Lean Kanban-based lifecycle (very small batches) or Lean Continuous Delivery (no batches) lifecycle. Thus, to successfully scale Agile, enterprises must work with business and Finance partners to move beyond traditional project-centric funding models and adopt a product-centric funding model that enables rolling-wave planning, dynamic resource allocation, and accelerated delivery.  Moving from funding project scope to funding teams is a seminal part of product centric funding.

However, a clear understanding and alignment on “Product” and “Product Portfolio” terminology is imperative before delving into the product-based funding model and financial governance.  It is often surprising to agile team members through all levels of leadership how hard it can be to all agree on what a “product” is.  We often start by getting an agreement on what a product isn’t, such as a platform or other part of the technology stack or a corporate function such as marketing.

What is a Product?

A product is designed to continuously create business value for the customer by solving their problem or providing a benefit. Products have more permanence, and are living entities that we continuously iterate to meet market needs and finally are retired when the demand for it diminishes.

On the other hand, a project is a temporary endeavor, with a clear definition of the work that needs to be delivered, within a defined budget and by a specified date in time.

Key characteristics of a Product and a Project are elaborated below:

So, what are the benefits of pivoting to a product mode?

High Performing Teams

Standing Projects up and down is Inefficient and runs the risk of disbanding teams just as they enter the norming or performing phase. Organizations often underestimate the staff onboarding costs and ramp up time. Most product-centric organizations try to keep the same people working on a product through the lifespan of the product. Overtime, these teams build the stakeholder relationships and business domain knowledge, being stable and long-lived can benefit from a long performing phase.

Maintain Strategic Focus

Projects are funded independently and investments tend to be quite scattered and fragmented. Often, this leads to executives not having enough confidence that much of their investments are committed to top strategic priorities. Moreover, it really slows the organization’s response to change in business priorities.

Product Roadmaps are aligned with business capabilities and deliver measurable business outcomes. Further, funding is continuous with frequent checkpoints, allowing to dynamically reposition investments should the business priorities change

Ability to Truly Iterate

Projects are funded in one go, the entire agreed upon scope is funded to deliver the projected benefits. These un-validated hypotheses of benefits in the business case are based on a lot of assumptions, and it is often not feasible to clarify the entire scope upfront, despite the significant investment routinely made with traditional approaches. The reality is that many projects regularly miss the mark in terms of delivering benefits, and organizations often don’t have an effective process in place to validate actual benefits post every release.

On the other hand, product funding is continuous and flow based on validated learning of short timelines. This is a truly iterative approach that allows to pivot or preserve strategy to maximize the value delivery.

Customer focused

Project teams measure success as the delivery of agreed up-front scope, on time and in budget. This often means that they get too solution focused and lose sight of whether appropriate value is being delivered to the stakeholders. There is no point in delivering the entire upfront agreed scope if it doesn’t cater to the stakeholder needs anymore.

We all work to create value for our stakeholders as well as the organization. Business outcomes allow us to define value in a measurable way, thus focusing on what matters most from the customer perspective. For product teams, success equals improvement directly related to a business outcome. Hence, rather than seeing their job as delivering a task, product teams focus on delighting and adding value for their customers (internal or external).

Knowledge Retention

Project teams ramp up quickly to build a solution over one or more releases, hand it over to an operations team in the “run” organization and are then disbanded, and the members move on to other project teams. With project teams being continually dragged onto new things, it gets very difficult to retain knowledge in legacy systems and often results in unmaintainable code, making it much harder to support such systems.

On the other hand, knowledge grows in product teams that allows team members to focus on a given business area for much longer. Overtime, these teams build strong stakeholder relationships and business domain knowledge, and can better understand the stakeholder problems and serve to their needs.

System Integrity and Continuous Improvement

Project teams are in constant pressure to deliver the agreed scope in defined timelines. This often leads to cutting corners and applying tactical fixes, increasing technical debt and neglecting long term architectural integrity in favor of short-term feature delivery. As the project team doesn’t face the long-term consequences of these tradeoffs, they are more likely to take such decisions for short-term gain. Over a period, this phenomenon compromises stability of systems, lowering quality and worsens the seed of value delivery.

On the other hand, Product teams have complete and collective ownership of the code and systems. There are no handovers, BAU or Operations team. The same team builds, runs and fixes any defects over the lifespan of the product, allowing them to evolve the system continuously and in a more sustainable manner.  This also fosters a mindset that promotes taking responsibility for their product and the decisions they are empowered to make.

Making the shift to a Product Centric Organization Model

According to a recent Gartner survey, 85% of the organizations have adopted or plans to adopt a product-centric organization model.

To enable accelerated value delivery, adaptive planning, and flexibility required to achieve the digital priorities, many organizations are shifting towards a product centric model. In a product centric model, organizations align funding & resources to product portfolios that support their key business capabilities.

A business capability is the ability of an organization to do things effectively to achieve desired business outcomes and measurable benefits. Each business capability is independent from other business capabilities and realized by combining different functions of an organization to fulfil one functionality. Example, for an FMCG, key business capabilities are often: Direct to Consumer, Wholesale and Retail.

The key primary constructs of translating Enterprise Strategy into traceable User Stories, via Features and Business Outcomes are:

• A Product Portfolio, also known as a Product Line, solely exists to fulfill its contribution toward realizing the overall enterprise strategy. Each Product line is tied to a distinct business capability with end to end accountability to deliver measurable business outcomes.

• The Portfolio Management board allocates funding across each of the Product Lines based on the importance of the business capabilities they support and the business outcomes they are expected to deliver. The Product Line budget is then split into individual product streams by the Product Line Council, which is then further split into individual feature teams by the Product Council.

• The business outcomes communicate aspects of strategic intent from the enterprise to the product line. Business outcomes are a means of concisely capturing business needs in a measurable way. Business outcomes most often require product functionality as well as supporting business activities (sales, marketing, promotions, delivery chain, etc.). The product teams work with leadership to establish measurable product outcomes tied to those business outcomes, which are then further split into Key Performance Indicators (KPIs) for each Product Feature team, who will then breakdown those features into user stories, establishing a clear line of sight from enterprise strategy to user stories.

Adaptive Funding of Products and Product Portfolios: From Annual to Quarterly Cycle

Organizing work by Products and Product Portfolios is a good starting point to accelerate the delivery of value to one’s customers. However, that enough won’t suffice if the work is still funded through projects on an annual basis.

An annual investment process simply can’t keep up with the pace of change. Under an annual planning approach, entire funding is allocated at the start of the fiscal year. Significant effort is spent upfront to create a detailed business case required to win the funding approval. Changes in stakeholder priorities or market demand often render a lot of early requirements captured in the business case as obsolete, creating a lot of waste and requiring significant rework.

Thus, to achieve their digital ambitions, enterprises must work with business and Finance partners to move beyond traditional project-centric funding models and adopt a product-centric funding model that enables rolling-wave planning, accelerated delivery of value, and validated learning to make much more informed decisions.  Moving from funding project scope to funding teams is a seminal part of product centric funding.

(Re)allocation of Funds across Product Portfolios

Initial annual funding allocations to individual product portfolios are based on un-validated hypothesis of benefits, which must be tested over the course of the year against changing priorities and proven value. The Portfolio Management Board meets on a recurring basis (Quarterly plus any market triggered event) to assess the allocation at the product-line level and reallocating funds as necessary to reflect changes to strategic enterprise priorities.

(Re)allocation of Funds within a Product Portfolio

The Product Line Council has the autonomy and the empowerment to reprioritize and reallocate funds across all products within the product line.  Each product team within a product line are allocated funds on a quarterly basis based on proven value (unless it’s the first quarter where funding is based on projected benefits). The business value is measured by hitting a set of KPIs tied to the business outcomes such as increasing revenues, reducing costs, and improving customer satisfaction. This level of decentralization in decision making is possible when an organization implements a consistent, standardized and predictable cadence to support governance, offering frequent opportunities for key stakeholders to provide input on the roadmap.

Impediments to adopt Product-based Funding Model

The shift from a project-based funding model to a product-based funding model requires a major cultural and mindset change, as different stakeholders will have different reservations about the new approach. Some of the challenges to adopt a product-based funding model include:

Business & Finance partners reluctant to cede control of budgets

Some CFOs and business leaders are reluctant to cede control of budgets. In response, they must be made to understand that the reality is that they are gaining a lot more control by not allocating the entire funding based on the un-validated hypothesis of projected benefits and with no benefits tracking in place most cases. Also, by offering them visibility into product portfolio performance on quarterly basis with the option to pivot or preserve funding strategy based on proven value and/or change in priorities, we can significantly reduce the risk of financial loss.

Product Managers lacking finance expertise to manage product budgets

Identifying Product Managers with sufficient expertise to manage large budgets is often challenging and requires significant effort and commitment from the organization to build the financial competencies.

Resistance to funding not” fully detailed” requirements or outcomes

Business partners feel a comfort factor with a detailed business case based on projected benefits, even if it is based on a lot of assumptions and uncertainty upfront. To get business partners fully on board, product leaders need to craft a pitch that provides tangible examples of how adaptive funding model leads to better business outcomes.

Requires structural changes to support product-centric organization

Most companies we work with are organized around functions. Organizational structure change is required to adopt a product-centric operation model. Organizations are reluctant to Org design changes as it can be very difficult and expensive to implement.

Parting Thoughts

We have endeavored to shed some light on a very complex topic, balancing the amount of context we can provide in a blog entry.  If you have any questions please do use the comments section below.

Joshua Barnes, CDAC | CDAI

Jeev Chugh, CDAP | CDAI

The Disciplined Agile (DA) Tool kit is a tremendous resource to be used in every Lean-Agile adoption / transformation effort, from team-level to enterprise-level. The DA Tool kit, now with more clarity on choosing your way of working (WoW), offers new and established teams an opportunity to evolve further through consideration of their context and the many goals, factors, and options available to them.

Disciplined Agile Delivery (DAD), with its focus on delivery has provided a solid foundation for building high performing delivery teams. As enterprises come to value the benefit of these team-level improvements the desire to leverage these improvements throughout the enterprise, scaling these improvements, is a common enterprise strategy today. Of course, DAD Teams are enterprise aware and ready to work with the enterprise to reduce overall delivery time and cost.

There are many scaling factors to consider as an enterprise seeks Agility at Scale. Additionally, the DA  tool kit provides straightforward guidance for scaling into Disciplined DevOps and then into value streams. Finally, the toolkit provides guidance for a Disciplined Agile Enterprise. It is here that I want to focus on Principle 6, Optimize Flow.

DA recognizes that enterprises are complex adaptive systems. The solutions they are providing are also complex solutions. Gone are the good’ol days of simply being complicated!

The challenge, as stated by the principle to Optimize Flow, is how to ensure collaborating teams do so in such a way as to effectively implement our organization’s value streams? And, how do we optimize our overall workflow?

I’ll take the liberty here to rephrase and combine the two questions. How do we organize long-standing teams to support the delivery of value for the enterprise in a way that allows for ongoing optimization experiments (as in Lean Change)?

To ‘implement’ the organization’s value streams for delivery of solutions, we must first identify and visualize the operational value stream of the enterprise and then determine the nature of the solution being delivered.

To the sharp-eyed reader, you’ve noticed that I’ve qualified the use of value stream with a reference to operational value stream. Yes, I believe there is a value stream that is characterized by the enterprise: The value stream through which it delivers value to its customers, clients, members, or whomever external to the enterprise is paying for the value delivered. This is the operational value stream, the flow from order to cash.

Well, there is no reason to qualify the use of value stream if not for at least a second type of value stream. That second type is a development value stream. The development value stream is where we can fully leverage the wealth of resources available from the DA Toolkit. This is not to say that we can’t use various ideas and principles from the DA Toolkit to improve the operational value stream. Only that the sweet spot for considering the Disciplined Agile Enterprise (DAE) is from the point of view of the development value stream.

So, what is a development value stream? A development value stream is responsible for implementation and maintenance of solutions that provide for business capabilities that are in turn needed to support to operational value stream. Here, in the development value stream, is where we are better able to apply the Optimize Flow strategies of delivery at a sustainable pace, optimization of the whole, making work flow, and using experiments to eliminate waste and to gain an understanding as those long-standing teams participating in the development value stream use validated learning through metrics to continuously improve.

A development value stream is implemented by way of establishing and organizing DAD teams in alignment with the flow of idea to solution for improvements of business capabilities within the enterprise.

Sustainable pace

For a team of teams aligned to enable business capabilities the continuous generation of business capability improvement ideas can easily exceed the capacity of the delivery teams to implement each idea. It is important the leaders within the enterprise understand that more can be accomplished with the people they have when they limit the amount of work requested of them to their capacity to do the work. Exceeding that capacity ultimately results in less work being accomplished overall as we lose productivity though excessive multitasking. Yet, each team will want to be responsive to the perceived demand of the enterprise. In their efforts to self-organize to improve on what they can accomplish it is important to keep the next strategy in mind, optimize the whole.

Optimize the whole

Within a team of teams each team is typically striving to contribute a smaller outcome that, together with the outcome of other teams, contributes to a larger outcome needed to improve a business capability. Teams should seek to optimize around the delivery of the larger outcome rather than only their own contribution. It is important to be aware that the optimization of a component of a system may, in fact, cause a sub-optimization of the larger system. Focus on the flow of the larger value delivery to make work flow better overall.

Make work flow

The principle to visualize work enables teams to identify and remove bottlenecks. Kanban is a great method for this visualization. But, the idea of a Kanban board is not limited to being applied only for a Kanban team. Scrum, and indeed, anyone or team can borrow from Kanban. An overarching visualization of the work is our need. For a team of teams, that visualization to optimize the whole needs to encompass the effort of the team of teams. While the individual teams may focus on the flow of user stories, the team of team should be visualizing the flow of the large outcomes that the team of teams is striving to delivery.  With that visualization the team of teams can work together to eliminate waste that, from their team point of view, might not be visible, or might not have been within their ability to influence improvement experiments that required the support of the team of teams.

Improve continuously

Everyone familiar with Scrum understands the value of periodic retrospectives. These retrospectives allow for the self-organization of the team to streamline their processes. This same approach should be considered for a team of teams. Periodically, perhaps once a quarter, the team of teams should collective gather to consider experiments that will streamline the more overarching issues they all face.

Experiment to learn

Consider the ‘implementation’ of all improvement ideas as experiments. Just as we can no longer provide concrete implementation plans for the complex technical solutions delivered today, we can not develop a concrete process/behavior/culture improvement plan for today’s enterprises that are themselves complex adaptive systems. Also, we’ve learned that small changes are better than big changes. Choose to learn with small Minimal Viable Changes (MVC). Advance your learning by leveraging the learning of Lean Change.

Measure what counts

For each experiment to learn determine the measures of success before starting the experiment. What is the goal of the change (experiment)? What questions need to be answered to know if the outcome of the experiment is likely to realize that outcome? What measurements can be used to obtain the information needed to answer those questions? Don’t waste time and money on vanity metrics.

Prefer long-lived stable teams

Where is the value once you optimize flow when you throw out that value by disbanding your high-performing team of teams? By aligning your team of teams, your development value stream, with business capabilities that enable your operational value stream you have created an organization that can be long-lived because the business capabilities of an enterprise tend to be persistent for the life of the enterprise.

The bottom line

When you find yourself coaching a larger number of teams for an enterprise with the vision to become more agile overall, I propose that you expand upon the principle of the Disciplined Agile Toolkit to Optimize Flow. The principle to Optimize flow applies at the larger scale of a team of teams. This article has hopefully helped you map the Optimize Flow principle to a larger number of teams, a team of teams, organized as a development value stream when you consider the transformation/adoption needed for a Disciplined Agile Enterprise.

In Time Tracking and Agile Software Development we overviewed why teams should consider tracking their time.  Primary reasons include:

• You’re billing your customer by the hour
• Your organization wants to account for CapEx/OpEx
• Your organization wants to take advantage of tax credits (typically for R&D work)

A secondary reason to track time is because the team wants to measure where they are spending time so as to target potential areas to improve.  This is more of a side benefit than anything else – if this was your only reason to track time you’d be better off simply discussing these sorts of issues in your retrospectives.  But if you were already tracking time then running a quick report to provide the team with intel likely makes sense for you.

So what are your options for recording time?  Potential strategies, which are compared in the following table, include:

1. Automated report from an agile management tool. The basic idea is to extract data from an agile management tool (JIRA, TFS, LeanKit, …) and load it into your time tracking system.
2. Manual input by team members. Each team member, typically once a week, inputs their time into the time tracking tool.
3. Manual input by the Team Lead. The Team Lead (ScrumMaster) inputs the time for their team into the time tracking tool.
4. Manual input by a Project Manager/Coordinator. A PM or Project Coordinator, often in a support role to the team, inputs the time of team.
5. Don’t track time at all. ‘Nuff said.

Table: Comparing options for tracking time.

This blog posting was motivated by a conversation that I had with Stacey Vetzal on Twitter.

Related Reading

• Time Tracking and Agile Software Development
• Agile Software Development and CapEx/OpEx

We are often told that agile teams should be whole, that they should have sufficient people, funding, and skills to fulfill their mission.  The idea is that this reduces the dependencies that your team has on others, enabling them to make decisions and to collaborate more effectively.  But, is this actually happening in practice?  Are agile teams truly whole, or do they still need to collaborate with other teams (hopefully productively) to get the job done?  Being strong believers in empiricism over rhetoric we decided to look into this issue.

In November of 2016 we ran the 2016 Agility at Scale survey.  It was targeted at people who were currently working on agile teams, or who had recently worked on agile teams, and we asked them straightforward questions around the size of the team, how distributed it was, what complexities they faced, an so on.  The following infographic summarizes the findings from the question that explored whether agile delivery teams need to work with external teams or groups to get their work done – in other words, are agile teams truly whole or do they rely on others?  As you can see, 96% of respondents indicated that in practice their team had to work with one or more other teams, leading to the conclusion that very few agile teams appear to be truly whole.

One of the fundamental principles underlying the Disciplined Agile (DA) toolkit is that disciplined agilists should be enterprise aware – they should recognize that they need to collaborate with others outside of their team, that they should work towards a common organizational vision, and that they should strive to do what is best for the organization and not just what is convenient for them.  Given that agile teams are collaborating with others in practice, it is clear that this philosophy of being enterprise aware is important.

The following diagram presents the results from the survey question in greater detail.  You can obtain the source data, a copy of the original questions, and a slide deck key diagrams at the 2016 Agility at Scale survey page.

Related Posts:

• Can you outsource and still be agile?
• Do agile teams take on hard problems?
• Do agile teams face regulatory compliance?
• How geographically distributed are agile teams in practice?
• How large are agile teams in practice?

One of the key aspects of a disciplined agile approach is to be enterprise aware.   The fundamental observation is that your team is only one of many in your organization, except in the case of very small organizations, and as a result should act accordingly.  This means that what your team does should reflect your organization’s overall business and technical roadmaps, that you should strive to leverage as much of the existing infrastructure as possible, that you should try to enhance the existing infrastructure, and that you should work with other teams appropriately so that your overall efforts are complimentary to one another.  This is a straightforward idea conceptually, but in practice acting in an enterprise aware manner can prove more difficult than one would initially think.

Over the years we’ve been asked by several customer organizations to help them to understand how to account for the expense of agile software development.   In particular, incremental delivery of solutions into production or the marketplace seem to be causing confusion with the financial people within these organizations.   The details of accounting rules vary between countries, but the fundamentals are common.  For public companies capital expenses (CapEx) are preferable because they can boost book value through the increase in assets (in this case a software-based solution) and increase in net income (due to lower operating expenses that year).  On the other hand, operational expenses (OpEx) are accounted for in the year that they occur and thereby reduce net income which in turn reduces your organization’s taxes for that year.   Furthermore, in some countries you can even get tax credits for forms of software development that are research and development (R&D) in nature.  In order to get properly account for the expenses incurred by software development teams, and potentially to earn R&D tax credits, you need to keep track of the amount of work performed and the type of work performed to develop a given solution.  Time tracking doesn’t have to be complex: at one customer developers spend less than five minutes a week capturing such information.  The point is that the way that a software developer’s work is accounted for can have a non-trivial impact upon your organization’s financial position.  This in turn implies that the need for agile developers to their track time is a fairly simple example of acting in an enterprise aware manner.

So, I thought I’d run a simple test.  On LinkedIn’s Agile and Lean Software Development group I ran a simple poll to see what people thought about time tracking.  It provided five options to choose from:

• Yes, this is a valuable activity (33% of responses)
• Yes, this is a waste of time (39% of responses)
• No, but we’re thinking about doing so (2% of responses)
• No, we’ve never considered this (18% of responses)
• I don’t understand what you’re asking  (5% of responses, one of which was mine so that I could test the poll)

The poll results reveal that we have a long way to go when it comes to working in an enterprise aware manner.  Of the people inputting their time more of them believed it was a waste of time than understood it to be a valuable activity.  When you stop and think about it, the investment of five minutes a week to track your time could potentially save or even earn your organization many hundreds of dollars.  Looking at it from a dollar per minute point of view, it could be the highest value activity many developers perform that week.

The discussion that ensued regarding the poll was truly interesting.  Although there were several positive postings, and several neutral ones, many more were negative when it came to time tracking.  Some comments that stood out for me included:

• It’s a colossal waste of time unless you’re billing a customer by the hour.
• We record time spent on new development work (as distinct from other tasks such as bug fixing in legacy code and so on) as this is capitalised as an asset and depreciated.
• I think the *most* pointless example was where the managers told us what we should be putting in.
• One day we will move past the “just do it” mentality and have some meaningful conversations and the reasons for what we do.
• In my experience time tracking is a massive waste of time. It’s a poor substitute for management.
• Why do you need to know more than the info available through Sprint Backlog, Sprint burndown and the daily standup?
• Some of my teams (I am SM for three teams) are skeptical about this. They do not think that keeping track of task hours this way will be any more useful than the daily standup reports.  And they do not believe that Management can resist the temptation to use task hours as a measure.

So what can we make of this?  First, it’s clear that delivery teams need a better understanding of the bigger picture, including mundane things such as tax implications of what they’re doing.  Second, it’s also clear that management needs to communicate more effectively regarding why they’re asking people to track their time.  To be fair, management themselves might not be aware of the tax implications themselves so may not be making effective use of the time data they’re asking for.  Third, management needs to govern more effectively.  Several people were clearly concerned about how management was going to use the time data (by definition they are measures) which could be a symptom of both poor communication as well as poor governance (unfortunately many developers have experiences where measures have been used against them, a failure of governance, and no longer trust their management teams to do the right thing as a result).  Fourth, some of the team-focused agile practices, such as burndown charts (or better yet ranged burndown charts) and coordination meetings may be preventing people from become enterprise aware because they believe that all of their management needs are being met by these practices.  Finally, many organizations are potentially leaving money on the table by not being aware of the implications of how to expense software development.

In Summary

Disciplined agilists are enterprise aware.  This is important for two reasons: First, you want to optimize your organizational whole instead of sub-optimize on project-related efforts; second, you can completely miss opportunities to add real value for your organization.  In the anecdote I provided it was clear that some agile developers believe that an activity such as time tracking is a waste, when that clearly doesn’t have to be the case.  Worse yet, although someone brought up the issues around capitalizing software development expenses early in the conversation a group of very smart and very experienced people still missed this easy opportunity to see how they could add value to their organization.  It makes me wonder if some of the agile rhetoric is getting in our way of being more effective as professionals (and, BTW, there are light-weight options for tracking time available to you).

Related Reading

• Strategies for Tracking Time on Agile Teams
• Agile Software Development and CapEx/OpEx