Disciplined Agile

In this blog posting, the latest in our ongoing disciplined agile program management series,  we overview the external workflows that a large delivery team is likely to be involved with.

Workflow With Other IT Teams

The following diagram overviews the major workflows that a disciplined agile program is associated with.  Note that feedback is implied in the diagram.  For example, where you see the Technology Roadmap and Guidance flow from Enterprise Architecture to Program Management there is an implied feedback loop from the program to the enterprise architects.  Also note that the workflows do not necessarily imply that artifacts exist.  For example, the data guidance workflow from Data Management could be a conversation with a data management person, it could be a concise description of data standards, or it could be detailed meta data – or combinations thereof.  A second example would be a program providing their development intelligence to the IT governance team through automated rollup of metric data via your organizations dashboard technology.

The following table summarizes the workflows depicted in the diagram.

The activities associated with these process blades are often very highly related. For example, in some organizations the activities associated with enterprise architecture and reuse management are fulfilled by a single group.  In other organizations some product management activities are performed by the portfolio management team and some by the enterprise architecture team.  Some organizations may choose to have a separate group for each process blade.  And of course the organizational structure will evolve over time as your various teams learn how to work with one another.  Every organization is different.

Program Management and DevOps

A common question that we’ve gotten is how program management is affected by DevOps. For example, you see in the diagram that Operations, Support, and Release Management (amongst others) are shown as things that are external to Program Management.  Remember that the focus here is on process, not on team organization.  For example, in organizations with a disciplined DevOps strategy in place it is very common to see program teams taking on the responsibilities of operating and supporting their own systems in production, and of doing the work to release their solutions into production.  In organizations without a strong DevOps mindset (yet), you are likely to find that operations, support, and release management are done by separate groups outside of your program team.  Context counts, and it’s good to have a process framework that is flexible enough to support the situation that you find yourself in.

Related Postings

• Large Agile Teams
• Why Do We Need Agile Programs?
• Disciplined Agile Program Management 101
• Disciplined Agile Program Management: A Goal- Driven Approach
• Disciplined Agile Program Management: Internal Workflow

Enterprise architecture, when performed in a disciplined agile manner, is an important enabler of agile software delivery.  This is true for several reasons:

1. Common architecture enables agile teams to focus on value creation.  A common enterprise architecture enables reuse across delivery teams.  When agile teams have high-quality assets – such as micro-services, legacy data sources, and frameworks – available to reuse they are able to focus on creating new value for their stakeholders and not on reinventing new versions of existing assets.
2. Common technical guidance enables greater consistency between teams.  When team follow effective, common conventions it results in greater quality.   This makes it easier to learn about assets that are new to them, in particular existing source code, and to evolve those assets as needed.  Greater consistency also makes it easier for people to move between teams because it will be easier for them to come up to speed on what the new team is doing and to share their skills with those team members.
3. Agile architectures enable disaggregation.  When your solutions are built from loosely coupled, highly cohesive components it is easier to spread development work across smaller teams.  This reduces overall risk and organizational complexity, which in turn reduces time-to-delivery.  It also allows agile teams to focus on what they’re building instead of coordinating with other teams or mocking out the (undelivered) work of other teams.
4. Common infrastructure enables continuous delivery.  When there is a common technical infrastructure to IT delivery teams to deploy into it is easier to deploy.  The easier it is to deploy, the more often it makes sense to deploy.
5. Enterprise architecture scales agile.  A disciplined agile approach to enterprise architecture enables organizations to scale agile strategies “horizontally” across their entire IT department.

This posting, the latest in a series focused on a disciplined agile approach to program management, overviews the activities associated with program management. The Disciplined Agile (DA) toolkit promotes an adaptive, context-sensitive strategy.  The framework does this via its goal-driven approach that indicates the process factors you need to consider, a range of techniques or strategies for you to address each process factor, and the advantages and disadvantages of each technique.  In this blog we present the goal diagram for the Program Management process blade and overview its process factors.

The following process goal diagram overviews the potential activities associated with disciplined agile program management.

The process factors that you need to consider for program management are:

1. Allocate work. Work items must be allocated to delivery teams, or to open source efforts in the case of programs which include internal open source components, throughout the lifecycle.  The type of work and the focus of the sub-team are the primary determinants of how work is allocated.  However, team capacity and load balancing concerns, for example a team has run out of work or a team currently has too much work, will also be considered when allocating new work.  Work allocation is the responsibility of your product owners although team capacity planning and monitoring is typically performed by the program manager and team leads.  Regardless, these activities should be performed collaboratively by the available people at the time.
2. Prioritize work.  The work performed by the teams – including new requirements and fixing defects – needs to be prioritized.  There are several ways to prioritize the work, such as by business value, by risk, by severity (in the case of production defects), or by weighted shortest job first (wsjf) to name a few strategies.  Prioritization is an ongoing activity throughout the lifecycle and is the responsibility of your product owners.
3. Plan program.  Traditional programs are often planned on an annual or even ad-hoc basis.  Agile programs, at least the disciplined ones. tend to be planning on a rolling wave basis.
4. Organize teams.  There are three common strategies for how you can organize delivery teams within a program – feature teams, component teams, and internal open source – each of which has advantages and disadvantages.  In addition to delivery teams, in a large program you are likely to find the need for leadership teams – the Product Owner team, the Architecture Owner team, and the Product Delivery/Management team – made up of the product owners, architecture owners, and team leads from the delivery teams respectively.  These leadership teams are responsible for work/requirements coordination, technical coordination, and management coordination within the program respectively.
5. Coordinate teams.  There are several ways that the sub-teams can coordinate with one another.  For example they could choose to have cross-team coordination meetings (also called a Scrum of Scrums (SoS)); they could visualize the work through task boards, team dashboards, and other information radiators such as a modeling wall; they could choose to have “big room” planning sessions where all team members are involved or “small room” agile modeling sessions where a subset of people are involved; or even traditional (or agile) checkpoint meetings.  All of these strategies have their advantages and disadvantages, and all can be applied by the various types of teams mentioned earlier.
6. Coordinate schedules.  There are several strategies that a program can adopt to coordinate the schedules between sub teams.  The easiest conceptually, although often hardest to implement in practice, is to have all sub-teams on the same cadence (e.g. every sub-team has a two week iteration).  This is what both SAFe and LeSS prescribe.  Another option is to have multiplier cadences where the schedules of sub-teams align every so often.  For example, we once worked with a large program where some sub-teams had a one-week iteration, some had a two-week iteration, and a few had a four-week iteration.  We’ve also seen another team where sub-teams had one, two, or three week iterations that provided alignment of iteration endings every six weeks.  Most common, although rarely discussed, is for sub-teams to have disparate cadences.  This is guaranteed to occur when teams are following different lifecycles (remember, the DA toolkit supports several).  For example, when some sub-teams are following the Scrum-based agile/basic lifecycle that has iterations, yet other sub-teams are following the lean or continuous delivery lifecycles that have no iterations, then you have an alignment challenge.  Or if you have sub-teams adopting any iteration length they like (we’ve seen some programs with sub-teams with two, three, four and sometimes even five week iteration lengths) then they also in effect have disparate cadences.
7. Schedule solution releases.  Programs need to schedule their own releases, in accordance to your organization’s release management strategy, which involves coordination between the sub-teams.  When the cadences of the sub-teams are (reasonably) aligned then it is easier to coordinate production releases.  For example, when all sub-teams have two-week iterations (or at least the sub-teams with iterations do) then they could potentially release into production every two weeks.  In the case of multiplier cadences, there is the potential to release into production each time the iteration endings align.
8. Negotiate functional dependencies.  An important responsibility of the Product Owner team is to manage the functional dependencies between the work being performed by various sub-teams.  There are strategies to manage dependencies between two agile sub-teams, between an agile sub-team and a lean sub-team, and even between an agile/lean sub-team and a traditional sub-team (this isn’t ideal, but sometimes happens).
9. Negotiate technical dependencies.  Similarly, an important responsibility of the Architecture Owner team is to work through technical dependencies within the solution being developed by the program.
10. Govern the program.  The program must be governed, both internally within the program itself while still operating under the aegis of your organization’s overall IT governance strategy.  Program-level metrics, particularly those tracking the progress of sub-teams and the quality being delivered, are vital to successful coordination within the program.  Sub-teams should also be working to common conventions, ideally those of the organization but in some cases specific to the program itself (perhaps your solution is pioneering a new user interface look-and-feel or new data storage conventions).  Programs, because of their size and because they are usually higher risk, often have more rigorous reporting requirements for senior management so as to provide greater transparency to them.  The implication is that a program’s dashboard often has a more robust collection of measures on display.

How Scaling Affects Program Management

Although program management primarily addresses the team size scaling factor, your tailoring decisions will still be affected by the other scaling factors:

1. Geographic distribution.  Chances are very good that large teams will also be geographically distributed in some way.  There are two flavors of this: Are teams geographically distributed (e.g. in different physical locations) and are people within a team geographically dispersed (e.g. people are working in cubes, on different floors, in different buildings, or from home)?  Both add risk. Coordination within the program becomes more difficult the more distributed the teams are, and more difficult within teams the more distributed the people are.  Distribution hits the leadership (the product owner team, the architecture owner team, and the team lead/product delivery) teams particuarly hard because members should be located with their delivery sub-teams but also need to work regularly with their counterparts located elsewhere.  The implication is that the team may require more sophisticated tooling to enable collaboration and more importantly be prepared to invest in travel regularly to foster better communication between disparate locations.  Furthermore, when your stakeholders are geographically distributed it may require your Product Owners to get support from agile Business Analysts in the various locations to help elicit requirements from them.
2. Compliance.  Compliance, either regulatory compliance required by law or self-imposed compliance (i.e. CMMI-compliancy), will definitely have an effect on your approach to program management.  In fact, the larger the program the more likely it is to fall under regulatory compliance due to the greater risk involved. Regulatory compliance generally requires greater governance both within the program and outwards facing as well.  Under some regulations your coordination efforts will require proof that they occurred, such as some form of meeting minutes that capture who was involved, the decisions made (if any), and action items taken by people.  Compliancy may also motivate more sophisticated approaches to capturing requirements by your Product Owners and to documenting technical concerns by your Architecture Owners.
3. Organizational distribution. The larger the team, the more likely you are to involve contractors, consultants, or even to outsource portions of the work.  When external organizations are involved the Program Manager will likely be involved it the contract management effort, which in turn may require assistance by the team leads.
4. Technical complexity.  The larger the team, the more likely it is that they are taking on greater technical complexity.  Or, another way to look at it, greater complexity often motivates the creation of larger teams to deal with that complexity.  Greater technical complexity will motivate greater attention to architecture and design, thereby motivating more regular collaboration of the Architecture Owners.
5. Domain complexity.  Similarly, team size and domain complexity tend to go hand-in-hand.  Greater domain complexity will require the Product Owners to work in a more sophisticated manner and may even motivate them to get support from agile Business Analysts (or junior Product Owners as the case may be).

Concluding Thoughts

In some ways “program coordination” is a more accurate term than “program management.”  Unfortunately “program management” is a far more common term within the IT community so we have decided to stick with it.

Our next blog posting in this series will describe the workflow of program management with other key IT activities such as portfolio management, enterprise architecture, and IT delivery teams.

Related Postings

• Large Agile Teams
• Why Do We Need Agile Programs?
• Disciplined Agile Program Management 101
• Discplined Agile Program Management: External Workflow
• Disciplined Agile Program Management: Internal Workflow

An IT program is a large team composed of two or more sub-teams.  The purpose of program management is to coordinate the efforts of the sub-teams to ensure they work together effectively towards the common goal of producing a consumable solution for their stakeholders.  In this blog posting we explore the team structure of a typical large agile program and the type of activities that program management addresses.

Team Structure of a Large Program

We described in detail the team structure of a large agile program in Large Agile Teams.  The key ideas are that a large team is organized as a team of teams and that structures are required to coordinate people, requirement, and technical concerns within the overall program.  Where a “scrum of scrums” may suffice for this coordination on small-to-medium sized programs (say up to five or six sub-teams), it quickly falls apart for larger programs.  As a result large programs will find that they need: a Product Management (or Product Ownership) strategy where the Product Owners coordinate their activities; an Architecture (or Architecture Ownership) strategy where the Architecture Owners coordinate their activities; and a Product Delivery (or Management) strategy where the Team Leads coordinate their activities.  The Program Manager, a specialist role, is responsible for coordinating the overall leadership team.

The following structure shows how the Product Owners of each sub-team are also members of the Product Management/Product Ownership team for the program.  Similar structures, see Large Agile Teams, will also exist for Product Delivery and Architecture as well.

The Program Lifecycle

DAD includes a Program lifecycle for a team of teams, depicted below.

What Program Management Addresses

There are several important concerns that program management needs to address:

• Architecture. The sub-teams must have a common architectural strategy that they are working to.  The initial architectural strategy will be identified early and will evolve throughout the lifecycle, requiring the architecture owners on each team to coordinate their efforts.  Although there are several ways to do this, a common strategy is for all of the AOs to hold  a weekly architecture meeting to work through program-level technical issues.   Subsets of the AOs will also hold impromptu meetings during the week to deal with immediate technical issues as needed.
• Work item management.  The sub-teams work together to develop a consumable solution for their stakeholders.  The implication is that they need mechanisms in place to allot the work between the sub-teams, including development of requirements, change requests, and fixing of defects.  The product owners on each team will need to coordinate their efforts throughout the lifecycle.  Common strategies to do this include the POs holding joint look-ahead modelling sessions (called backlog grooming/refinement sessions in Scrum) with key stakeholders; work coordination sessions where they identify dependencies between requirements and match the work to the sub-teams; and impromptu sessions between a subset of the POs throughout an iteration to work through dependencies between work being performed by different sub-teams.
• Management coordination between sub-teams.  The sub-teams will need to coordinate “management issues” with one another on a regular basis.  These management issues may include conflict resolution between people on different teams, enabling people to move between teams, financial tracking across teams, and many other concerns.  As a result the teams leads, and any appropriate team members involved with a given issue, will need to meet regularly to address any issues.  A common mechanism for this is for the team leads to hold their own daily coordination meeting after their team’s coordination meeting, a strategy that is often referred to as a “scrum of scrums”.
• Sub-team cadences.  The Disciplined Agile tool kit, unlike most agile scaling frameworks, does not insist that sub-teams follow the same iteration cadence (or even that sub-teams follow an iteration-based lifecycle).  For example, one sub-team may have a one-week iteration length, five sub teams have a two-week iteration length, and two sub-teams follow DAD’s continuous delivery life cycle which doesn’t have iterations at all.  Or you may find a program where every sub-team has two-week iterations.  Or a strategy where all teams take a continuous delivery approach.  Each of these strategies make sense in certain situations, but none of them make sense in all situations.  There is no single “best practice“.
• Release cadence.  There are two fundamental issues here.  First, how often will the overall program release into production?  Will it release monthly?  Quarterly?  Bi-annually?  Something else?  Second, how often will the sub-teams release their work internally to other sub-teams so that it may all be integrated and tested as a whole?  At the end of each iteration?  At the end of each day?  Several times a day?
• Governance.  The program will be governed.  This governance is both inward facing, the program should govern itself, and be outward facing, the program is part of your organization’s portfolio and therefore will be governed as part of your organizations overall IT governance strategy.

Relationship with Other Process Blades

There is clearly overlap with some of the activities in the portfolio management, enterprise architecture, release management, product management, and governance process blades.  The issue is one of scope.  Where these process blades address activities across all of your organization, the scope of the related activities within program management is the program itself.  For example, where enterprise architecture addresses architectural issues for the entire organization, the architecture activities encompassed by program management relate only to the architecture of the solution being produced by the program.

Related Resources

• For an understanding of the team structures within a program, read the article Large Agile Teams.
• The Disciplined Agile Program Management Blade

On June 25, 2015 we gave a webinar entitled Disciplined Agile Outsourcing: Making it Work for Both the Customer and the Service Provider (click on the link to see the slides on Slideshare).  We will likely run the webinar again as it was very well received.  We had hoped to record it but unfortunately weren’t able to save the file because the software crashed.  Hopefully it will work out next time.  During the webinar we promised to write a blog summarizing it and addressing any remaining questions from the audience.  This is that blog.

Agile and Outsourcing?  Really?

There are good reasons why organizations are taking an agile approach to outsouring IT delivery projects.  The following table overviews the motivation and concerns of both customer organizations and service providers.

Table 1. Comparing the Viewpoints of Customers and Service Providers

Here are some interesting statistics about outsourcing:

• Gartner’s 2014 IT Outsourcing report estimated IT outsourcing to be $288 billion in 2013 and to grown by 5.9% compounded annually from 2013 through 2018.
• Deloitte’s 2014 Global Outsourcing survey found that 53% of organizations are outsourcing some of their IT function; 26% of respondents who do not outsource today plan to do so (supporting Garter’s growth forecast); 79% of respondents DO NOT believe their service providers are too expensive; 49% of respondents say their service providers are reactive vs proactive (e.g. they are basically order takers, not partners); and IT outsourcing activity is expected to increase.
• The Ambysoft 2013 Agile Outsourcing survey found that half of organizations who are “doing agile” are also involving outsourcing in some way

The Challenges Associated with Agile IT Outsourcing

Here are some common pitfalls that organizations run into with IT outsourcing:

• Organizational culture differences
• Expectations mismatch between the customer and the service provider
• The customer underestimates difficulty of managing outsourced projects
• Total cost of the solution isn’t considered
• Total value of the solution isn’t considered
• Transition to the operations team is mismanaged
• Over-reliance on documentation
• Software licensing issues
• Learning curve for service provider underestimated
• The service provider is understaffed
• Some aspects, e.g. security, cannot be outsourced
• Intellectual property (IP) rights
• Technology connectivity
• Solution doesn’t fit into organizational ecosystem

Here are some common pitfalls with agile IT outsourcing:

• The customer procures the “agile” project via traditional strategies
• The customer takes a Water-Scrum-Fall approach
• The customer governs the service provider via a traditional approach
• The customer really isn’t agile
• The service provider really isn’t agile
• Neither are agile
• Agile is based on trust, yet it behooves you to not trust the service provider

Here are some common pitfalls with geographically distributed teams:

• Project success rates drop as geographic distribution increases
• Communication challenges
• Time zone differences
• Cultural differences
• Customer unwilling to invest in travel

How Disciplined Agile Delivery Supports Outsourcing

Disciplined Agile Delivery (DAD) provides several important mechanisms which enable agile IT outsourcing:

1. Flexible, process goal driven approach.  A team where the work has been outsourced to them will work differently than an insourced team.  A globally distributed team will work differently than a co-located team.  To be successful at agile IT outsourcing you need to adopt a process that is sufficiently flexible to meet the needs of the situation that you face.  DAD’s process goal-driven strategy provides this flexibility while being easy to tailor.  Context counts.
2. Hybrid framework.  The DA toolkit is a hybrid that adopts strategies from a variety of sources, including Scrum, Extreme Programming, Agile Modeling, Unified Process, Kanban, SAFe, traditional sources, and many more.  As a result it provides the robust advice that you require to address the complexities of agile outsourcing.  Choice is good.
3. Full delivery lifecycles. The DA toolkit supports four full delivery lifecycles – An agile Scrum-based on, a Lean Kanban-based on, an exploratory Lean Startup lifecycle, and a continuous delivery lifecycle – that show how to approach IT delivery from beginning to end in a streamlined and lightweight manner.
4. Risk-value lifecycle.  The DA toolkit explicitly includes proven risk reduction techniques in its delivery lifecycles.  These strategies include coming to a stakeholder vision early in the lifecycle, proving the architecture with working code, and delivering a potentially consumable solution on a regular basis.
5. Enterprise awareness.  The DA toolkit promotes the philosophy of enterprise awareness – DAD teams should do what’s best for the customer organization, not what is convenient for them.  This means that delivery teams should adopt organizational conventions whenever appropriate, reuse existing assets (including legacy data sources), follow the organization’s business roadmap, and follow the technology roadmap.
6. Explicit governance.  The DA toolkit explicitly address how to govern agile teams.  The increased risks associated with outsourcing require a more sophisticated approach to governance than what you would require for an insourced delivery team.

Disciplined Strategies to Make Agile Outsourcing Work

Our experience is that the reason why many customer organizations struggle with agile outsourcing is because their intuition (which typically reflects the traditional mindset) leads them astray.  The following table indicates what traditional intuition often leads you to do and the corresponding disciplined agile strategies that we recommend.

Table 2. Traditional intuition vs. what you should actually do.

When Disciplined Agile Outsourcing Makes Sense

We’ve found that for agile outsourcing to be successful you must first meet all three of the following criteria:

1. You are already successful at insourced agile.
2. You understand and accept the risks involved with outsourcing.
3. You are prepared to address those risks.