Disciplined Agile

This posting, the first in a series, overviews a disciplined approach to DevOps.  It begins by defining DevOps, no small task given the continued debate within the DevOps community, and then described a disciplined approach to DevOps.

Defining DevOps

For our purposes we propose the following definition:

DevOps is the streamlining of the activities surrounding IT solution development (dev) and IT operations (ops). 

Figure 1 below summarizes this idea, showing how the development and operations groups within your IT department begin to breakdown the barriers between them that inhibit effective collaboration.  In organizations that have not yet adopted a DevOps mindset we say that there is a “DevOps gap.”  This gap results in lengthy solution deployments and hence higher costs to deploy; a long mean time between deployments (MTBD) which is often measured in terms of months; reduced market competitiveness; and reduced ability to govern your IT efforts due to lack of real-time intelligence.  When organizations close the DevOps gap, by adopting strategies described later in this article, they effectively address these challenges and more.  They do this by adopting a mindset that promotes collaborative, learning-centric ways of working that are supported by agile practices and automation.

Figure 1. Closing the gap between IT software development and IT operations.

Defining Disciplined DevOps

What does it mean to take a disciplined approach to DevOps? It requires discipline to do the things that are good for you that you may not normally choose to do.  Given this, we propose the following definition:

Disciplined DevOps is the streamlining of IT solution development and IT operations activities, and supporting enterprise-IT activities, to provide more effective outcomes to an organization.

To understand the difference, let’s work through the scope implications of what DevOps could mean to your organization.  Figure 2 depicts an initial strategy for scoping your DevOps efforts, basically rethinking the interactions between your Development teams and your IT Operations team.  Notice how in Figure 2 we’ve started to use the term “IT Operations” and not just Operations.  This is because many organizations choose to distinguish between their IT operations activities and their business operations activities (such as selling, distribution, marketing, and so on).  Of course IT operations exists to support business operations.

Figure 2. An initial view of DevOps.

Some organizations will start with the strategy of Figure 3, particularly when there is a separate group responsible for managing the release/deployment of systems into production.  These Release Management teams are often, but not always, a subgroup of your IT Operations team.   Product companies that are creating solutions for the marketplace will typically have a separate Release Management team because the release effort includes both IT release activities, in particular releasing updated solutions into production, as well as customer-facing release activities (such as marketing, sales, and so on).  A disciplined DevOps strategy streamlines both the IT and the business aspects of your Release Management activities.

Figure 3. DevOps with an explicit Release Management activity.

Although Figure 3 is clearly a step in the right direction, it isn’t complete.  A Disciplined DevOps strategy will streamline the supporting activities performed by your Enterprise Architecture and Data Management teams.  Enterprise architecture activities that support DevOps includes the definition, support, and evolution of technical and business roadmaps which guide the efforts of the rest of the organization.  This in turn supports the creation of a common and consistent technical infrastructure within your production environments, enabling common DevOps practices such as continuous deployment, automated end-to-end regression testing, and operational monitoring.  Supporting data management activities include the definition, support, and evolution of data and information standards and guidelines; the creation, support, evolution, and operation of data sources of record within your organization; and the creation, support, evolution, and operation of  data warehouse (DW)/business intelligence (BI) solutions.  All of these activities will be described in greater detail later in future blog postings.

Figure 4. Disciplined DevOps – An enterprise view.

All three views are valid, your organization needs to decide which one is most appropriate for them given their current situation.  Figures 2 through 4 have been presented from the point of view of a potential DevOps maturity model – you could start with the initial strategy presented in Figure 2, then improve it to address release management (Figure 3) and finally evolve your strategy to address other IT activities such as enterprise architecture and data management (Figure 4).

In following blog postings we will show how DevOps strategies are explicitly addressed by the Disciplined Agile (DA) toolkit, thereby taking some of the mystery about how DevOps works in practice.

We are often asked what makes a Disciplined Agile (DA) coach effective.  The features to look for in a DA coach include:

1. People skills.  First and foremost, effective DA coaches need solid people skills.  They need to be prepared to work with a wide range of people coming from different backgrounds, with different learning preferences, and with different learning goals.  As a result DA coaches need to be empathetic, patient, respectful, and open-minded.
2. Experience.  Good coaches understands the situation that you face, and more importantly understands how to guide you to tailor your strategy.  Effective coaches have many many data points from their experiences over the years, and as such they can recognize patterns quickly and provide appropriate advice to those their are coaching.  We’ve seen people who are very good agile coaches for small, co-located teams get into serious trouble the first time they need to deal with scaling factors such as large team size, geographic distribution, or regulatory compliance.   We’ve also seen good development team coaches flounder when they first start to address, in a meaningful way, the Agile IT issues faced by organizations applying agile across their entire IT department.  This is one of the reasons why we suggest that Transformation coaches be Certified Disciplined Agile Coaches (CDACs) as they need significant experience and knowledge to be successful.  The implication is that when you’re hiring a coach, make sure they’ve worked in environments similar to yours otherwise you run the risk of paying for their learning experiences.
3. Pragmatism.  As Mark Lines astutely pointed out a few months ago, DA is pragmatic agile.  Effective DA coaches are willing and able to work closely with the people that they’re coaching, providing practical advice that they follow themselves. They also like to have real-time measures that reveal how their team(s) are doing, enabling them to make fact-based suggestions to help their teams.
4. Knowledge.  It’s reasonable to expect a DA coach to be very knowledgeable about DA and agile in general.  Development team coaches are at least Certified Disciplined Agile Practitioners (CDAPs) and better yet CDACs.  To earn a CDAP you need to have at least two years agile experience, clearly a bare minimum for someone in a coaching role, and be able to pass a challenging test which explores their understanding of the DA toolkit.  CDACs need to have five or more years of experience and must pass a board-level interview.  These are meaningful certifications that people must work for to earn, and are a clear indication that holders of such certification have the requisite DA knowledge.   Transformation coaches, who coach an organization’s executive team through the process of transitioning to agile, should be CDACs.
5. Skill.  Development team coaches must be skilled in fundamental agile techniques such as regression testing,  continuous integration (CI), iteration/sprint planning, look-ahead modelling and planning, requirements envisioning, and many more.  A good team coach should also be skilled in “advanced” agile techniques such as test-driven development (TDD), behaviour driven development (BDD), and continuous deployment.  Transformation coaches should be skilled in organizational change management as well as the fundamentals of IT-level activities such as enterprise architecture, data management, operations, portfolio management, and others.
6. Leadership.  In addition to solid people skills, good coaches often need good leadership skills too as they need to be adept at convincing people to follow their advice.  Team coaches will often be Team Leads, or at least be working closely with the Team Lead, to help lead the team in making the “hard decisions” required to successfully learn the agile mindset.
7. Flexibility.  A fundamental concept of the DA toolkit is that you need to tailor it to meet the needs that you face.  The implication is that DA coaches need to be agile to go beyond the advice in prescriptive methods such as Scrum or SAFe.  Instead of working from a prescriptive playbook, DA coaches will leverage DA’s goal-driven strategy to help guide teams make process-oriented and organization-oriented choices that are right for them. In short, just because someone has several years of Scrum coaching you can’t count on them having the background to be a good DA coach because they may only understand Scrum strategies and not the full range of agile and lean strategies supported by the DA toolkit.

DAD incorporates a number of milestones into its lifecycles to gauge the progress and health of your DAD projects/releases.

Why does DAD have milestones?

Governance is largely absent from mainstream agile methods and many believe that agile does not need milestones.   Scrum orders its backlog with highest value work at the top of the stack.  At the end of each sprint (iteration) a decision is intended to be made whether or not the next sprint of work in the backlog provides sufficient value to fund another sprint.  If not, the project stops.  While in principle this strategy makes sense, as a replacement for governance it is flawed for several reasons:

• Not sufficient.  There is much more to effective governance than deciding when to cease the funding of a project.  How did it get started in the first place for instance?
• Not realistic.  In reality most organizations tend to fund releases comprised of many iterations rather than make explicit funding decisions at the end of each sprint.
• Effectively a milestone.  While Scrum claims to not have milestones, pausing to consider if sufficient functionality exists to deploy, and to determine a go-forward strategy is implicitly a milestone.  DAD makes them explicit with guidance for how to effectively apply them.

Why do we need agile governance?

Contrary to what many believe, the need for governance does not disappear for agile initiatives.  Sponsors and other stakeholders deserve and indeed demand periodic updates on the status of their investments.  They want answers to questions like:

• What will I get from my investment and will it be worthwhile?
• Is what is being delivered consistent with the initial funding request?
• What is the status of the outstanding risks and what is being done to mitigate them?
• Is the current release timeline consistent with the original projection?
• When will we have sufficient functionality to go live?
• Are all stakeholders prepared to receive the release?
• When will the project be finished?

Claims that agile is “different” and does not need to provide these answers is one of the reasons that agile is sometimes seen as being undisciplined.  In fact these claims are often used by some as an excuse why agile won’t work for them.  Fortunately, since DAD has built in mechanisms to facilitate answering these questions many organizations are extending their agile implementations to incorporate DAD’s guidance.  Indeed DAD is very appealing to those looking for a way to govern projects while still remaining agile.

Milestone Reviews Should Be Kept Informal

Consistent with the strategy recommended in the DAD goal Govern Delivery Team, milestone reviews should be done in a light manner.  They typically coincide with the end of iterations or phases when when using the Agile/Basic lifecycle.  If you are unsure why DAD has phases you might find the blog posting on the rationale interesting.

Milestone Dates Should Be Communicated

It is a good practice to provide a calendar to stakeholders with milestone dates for each DAD project so that they can plan to attend iteration reviews that coincide with milestone reviews that they are interested in.  While milestone dates may change, it is very useful to have target milestones across a portfolio of projects visible as a roadmap to all stakeholders.

The DAD Milestones

Stakeholder Vision.  The goal of the Inception phase is to spend a short yet sufficient amount of time, typically one to four weeks in order gain stakeholder agreement that the initiative makes sense and to therefore continue into the Construction phase.  By addressing each of the DAD Inception goals the delivery team will capture traditional project information related to initial scope, technology, schedule, budget, risks, and other information albeit in as lightweight a fashion as possible.  This information is consolidated and presented to stakeholders as a Vision statement as described by the Develop Common Vision goal.  The format of the vision and formality of review with vary according to your situation.  A typical practice is to review a short set of slides with key stakeholders at the end of the Inception phase to ensure that everyone is on the same page with regard to the project intent and delivery approach.

Proven Architecture.  Early risk mitigation on a project is a part of any good project management discipline.  As the Prove Architecture Early process goal indicates, there are several strategies you may choose to adopt to do so.  The most effective of which is to build an end-to-end skeleton of working code that implements technically risky business requirements.  A key responsibility of DAD’s Architecture Owner role is to identify risks in the Inception phase.  It is expected that these risks will have been reduced or eliminated by implementing related functionality somewhere between one and three iterations into the Construction phase.  As a result of applying this approach early iteration reviews/demos often show the ability of the solution to support non-functional requirements in addition to, or instead of functional requirements.  For this reason it is important that architecture related stakeholders are given the opportunity to participate in these milestone reviews.

Continued Viability.  An optional milestone to include in your release schedule is related to project viability.  At certain times during the project stakeholders may request a checkpoint to ensure that the project is progressing consistent with the vision agreed to at the end of Inception.  Scheduling these milestones ensures that stakeholders are aware of key dates wherein they should get together with the team to assess the project status and agree to changes if necessary.  These changes could include anything such as funding levels, team makeup, scope, risk assessment, or even potentially cancelling the project.  There could be several of these milestones.  If the stakeholders agree to changes the Vision may be updated at this time.  Scrum implicitly has this milestone at the end of each sprint.  DAD makes this an explicit choice and makes it’s frequency optional.

Sufficient Functionality.  While it is worthwhile pursuing a goal of a consumable solution (what Scrum calls a potentially shippable increment) at the end of each iteration, it is more common to require a number of iterations of Construction before the team has implemented enough functionality to deploy.  While this is sometimes referred to as a minimally viable product (MVP) this not technically accurate as classically an MVP is meant to test the viability of a product rather than an indication of minimal deployable functionality.  The more accurate term to compare to this milestone would be “minimum feature set”.  Regardless, many use the acronym to mean the latter.

DAD’s sufficient functionality milestone is reached at the end of the Construction phase when a sufficient functionality is available plus the cost of transitioning the release to stakeholders is justified.  As an example, while an increment of a consumable solution may be available every two week iteration, it may take two weeks to actually deploy it in a high compliance environment so the cost of transitioning the functionality may not be justified until a greater amount of functionality is completed.

Production Ready.  For non-trivial situations a formal Transition phase is often required to do final preparations as part of delivery of the solution to stakeholders.  Once sufficient functionality has been developed and tested, transition related activities such as data conversions, final acceptance testing, production and support related documentation normally need to be completed. Ideally much of the work has been done continuously during the Construction phase as part of completing each increment of functionality. At some point a decision needs to be made that the solution is ready for production.  Of course, if you’re following the continuous delivery lifecycle then the “Transition phase” has evolved into a “Transition activity” – regardless, someone still needs to ensure that the solution is consumable before you deploy the solution.

Delighted Stakeholders.  Governance bodies and other stakeholders obviously like to know when the initiative is officially over so that they can begin another release or direct funds elsewhere.  The initiative doesn’t end the day after deploying a solution to stakeholders.  There are often project closeout activities such as training, deployment tuning, support handoffs, post implementation reviews, or even warranty periods before the project is considered completed.  Lean has a term “delighted customers” which suggests that “satisfied” customers is setting the bar too low.  While DAD agrees, there are more stakeholders than just customers such as production support that we need to delight before we consider the project complete, thus the milestone name “delighted stakeholders”.

Give Milestones a Try

Agile promotes transparency and accountability to our stakeholders.  Take this to the next level by sharing and celebrating achievement of key milestones on your projects.

Looking for some great DAD video content?  Trying to find just the right video to show your boss why moving beyond Scrum to DAD makes sense?  Well we are happy to announce that there is now a DAD YouTube Channel which can be found here:  https://www.youtube.com/channel/UCcWJ20C86Mzxcsqb73AReHQ  Subscribe to keep up to date on the latest DAD talks and webinars.  There is also a new link to this channel at the DAD blog website on the sidebar.

We are looking for multilingual presentations so if you are aware of any let us know.  For now we have added a talk in Russian.

When it comes to organizing large or geographically distributed agile teams many people will tell you that there are two strategies, taking what is called a component team approach or taking a feature team approach.  Many people will tell you to take a feature team approach over a component team approach, but the fact is that both strategies has advantages and disadvantages and neither is right for all situations.  Furthermore, those are the only two strategies as you will soon see, although to be fair they are the most common two.

This article explores the four basic strategies for organizing agile teams.  We compare and contrast these four strategies in Table 1 below, in accordance to the “it depends” philosophy of the Disciplined Agile Delivery (DAD) framework.  Our goal is to make it clear that you have choices when organizing agile teams, and that you should understand the trade-offs that you are making with each choice.  You will also find that you want to combine strategies, and in fact most large agile programs that we’ve seen do in fact do this.  The four strategies are:

1. Component teams. With this approach each sub-team is responsible for one or more subsystems or modules, something that can be difficult if some of your team works alone from home, to reduce the amount of information sharing and collaboration required between disparate teams.  Because component teams are effectively organized around your architecture you will need to invest sufficient time up front to identify that architecture.
2. Feature teams. A feature team is responsible for implementing a functional requirement, such as a user story or use case, from end-to-end.  This is often referred to as implementing a vertical slice of the solution.  Sometimes a given feature team will focus on the requirements for a single line of business (LoB), such as brokerage or life insurance within a financial institution, enabling them to gain expertise in that LoB. Other times a feature team will take on requirements specific to a given application or system within your organization.
3. Functional teams. Some large teams will be organized by development function – there is an architecture team, a development team, a testing team, a deployment team, and so on.  Each team focuses on their specialized function and hands off their work to other functional teams.
4. Internal open source teams. Sometimes a component or subsystem will be developed via an open source method, even though all of the work is private to your organization.  Developers from other teams voluntarily work on the component to evolve it over time.  When Scott was at IBM he saw this strategy work in practice for several important components reused within several IBM products.  For some detailed thoughts on strategy, read Reuse Through Internal Open Source.

Table 1. Comparing the team organization approaches.