Disciplined Agile

A fundamental philosophy of agile is that teams should be allowed to own their process, to choose their way of working (WoW). In Disciplined Agile (DA) we take it one step further with the idea that teams should not only be allowed to choose their WoW, they should be supported and enabled in doing so.  In other words, let’s help teams to be awesome.

Figure 1 depicts the workflow for how a team can choose and then evolve their WoW. The workflow shows four key activities that a team will iteratively work through as they need. In this blog we focus on initially choosing your team’s WoW.

Figure 1. The workflow for choosing and evolving your WoW (click to enlarge).

When adopting your initial WoW, the first thing to do is to identify whether you team is allowed to choose its WoW or whether one has been chosen for them. Let’s start by exploring how you would proceed if you’re allowed to choose your own WoW.

Tailoring Your Initial WoW

When a team is allowed to choose its own WoW the first step is to select the appropriate lifecycle given the situation faced by the team. Lifecycles, in some ways, are “methods” in that they show the high-level workflow for a team. They are the glue that combines detailed techniques/practices into a coherent whole (more on this below). Disciplined Agile Delivery (DAD) supports several lifecycles:

1. Agile. A Scrum-based project lifecycle.
2. Lean. A Kanban-based project lifecycle.
3. Continuous Delivery: Agile. A Scrum-based lifecycle for long-standing product teams.
4. Continuous Delivery: Lean. A Kanban-based lifecycle for long-standing product teams.
5. Exploratory. An experimentation-based lifecycle, based on Lean Startup, for exploring marketplace needs.
6. Program. A lifecycle for a large “team of teams.”

Although the focus of DA is on agile and lean ways of working, DA recognizes that in some cases you may still decide to adopt a waterfall/serial lifecycle. DA doesn’t explicitly support waterfall, but as you can see in Figure 2 we do recognize that in very low-risk situations a traditional approach makes sense.

Figure 2. A flowchart summarizing how to choose a lifecycle (click to enlarge).

Your lifecycle will of course evolve, either incrementally via normal evolution of your WoW or because your team explicitly decides to adopt a different one (once they’ve learned more about themselves as a team).

Once your team has chosen an initial lifecycle, the next is to select the detailed techniques that you’re going to follow as a team.  This is typically done as a series of process tailoring workshops where the team works through the appropriate goal diagrams to identify how they want to work together.  Figure 3 depicts the goal diagram for Secure Funding, you can see how it walks you through the decision points that you need to consider and potential techniques for addressing those decision points. Don’t worry, if you’re not familiar with all of the options they are described in the book Choose Your WoW!, with a description and the trade-offs associated with each technique summarized in tables. Knowing your options, and the trade-offs associated with them, enables your team to make better process decisions (which in turn leads to better process outcomes). This is something we call guided continuous improvement.

Figure 3. The Secure Funding process goal (click to enlarge).

In theory it’s possible to do a single “big bang” process tailoring workshop when a team is in initially formed, but we’ve found that leads to process bloat because the team has to guess at too many things all at once.  Unless you’re in a regulatory environment requiring defined process descriptions up front, it’s usually better to tailor your WoW in stages on an as-needed basis.

Adopt Existing Method or Framework

In some organizations teams are still not allowed to choose their WoW.  This may happen for several reasons:

• The organization is new to agile.
• Your organizational leadership wants every team to follow the same process (this is a spectacularly bad idea because context counts). This is often because they don’t understand that you can have a consistent governance process without inflicting the same process on all the teams.
• You’ve hired coaches that only know one method or framework.
• Leadership still has a command-and-control culture or they don’t trust your team to do this (either because you don’t have the experience required on your team or because you’re not using something like the DA toolkit yet).
• You’re in a regulatory environment (e.g. medical device development) and don’t realize that you can choose and evolve your WoW in any way that you like, as long as you remain compliant and document what you do (yes, that’s annoying).

The problem with forcing a team to follow an existing method or framework, no matter how popular it is, is that it rarely fits the situation faced by the team. It might be a great method, but it’s the wrong one for the team – context counts, every team is unique and faces a unique situation, so should be allowed to choose and evolve their own WoW to enable them to be as effective as they can be. Think of it like this: If a team is competent enough to build a solution for their stakeholders, surely they must also be competent enough (perhaps given a bit of help) to choose their own WoW?

Regardless of whether you were allowed to choose your initial WoW or had one forced upon your team, this is only a start. Your team will still want to evolve your WoW as you learn and as your situation evolves. More on this in our next blog.

MORE INFORMATION

For more information about how your agile team can choose and evolve its way of work, we recommend our book Choose Your WoW! A Disciplined Agile Delivery Handbook for Optimizing Your Way of Working. If you want to succeed at enterprise agile you need choices, not prescriptions.

In Continuous Improvement Through Experimentation we described how a team could improve their way of working (WoW) via the strategy depicted in Figure 1.  In Better Decisions Lead to Better Process Outcomes we showed how you can increase the rate of improvement by identifying potential improvements that are likely to work in your situation.  In this blog we describe how to accelerate your team’s improvement by running multiple improvement experiments in parallel.

Figure 1. Running an experiment to evolve your WoW (click to enlarge).

The strategy is exactly as it sounds.  Instead of running a single process improvement experiment at a time you run several at once.  You may decide to do this for one of several reasons:

• You want to compare several strategies that potentially address the same process issue you hope to resolve.
• You have several process issues that you hope to resolve at the same time.
• Your team is currently running an experiment and feels it has capacity to take on another improvement experiment.
• Techniques are dependent on one another, such as automated regression testing and continuous integration (CI).

Although this strategy will help your team to increase its rate of improvement, there are two fundamental challenges with it.  First, it is harder to assess the effectiveness of an individual technique when multiple experiments are being run in parallel because they can interact with each other.  For example, you decide to experiment with holding stakeholder demos and with active stakeholder participation at the same time (you weren’t doing either one until now).  When you see improvements in the quality of stakeholder satisfaction with the product you’re developing, is that the result of one of the practices but not the other or both of them in combination? If can’t definitively answer that question, it will make it hard to assess the effectiveness of each technique.

Second, you only have so much capacity for experimenting with new WoW. It can be hard enough, sometimes, to convince your organization’s leadership to allow you the time to experiment in the first place.  Getting time to run multiple experiments is even harder.

Our point is that running an experimentation-based approach to evolving your way of working (WoW) makes sense.  In some cases running multiple experiments in parallel can be even more effective.

MORE INFORMATION

For more information about choosing and evolving your WoW, we humbly suggest that you consider reading our book Choose Your WoW! A Disciplined Agile Delivery Handbook for Optimizing Your Way of Working. If you want to succeed at enterprise agile you need choices, not prescriptions.

In our previous blog, Continuous Improvement Through Experimentation, we worked through how teams can evolve their way of working (WoW) through experimentation and kaizen.  Figure 1 depicts the logic of a single pass through a Plan-Do-Study-Act cycle for doing so. The team identifies a potential way to improve their WoW, they experiment with it for a bit, they assess how well it worked for them, then they keep what works well and drop what doesn’t.

Figure 1. Continuous improvement through experimentation (click to enlarge).

Figure 2 shows how the effectiveness of a team’s WoW rises over time via this experimentation-based strategy.  When an experiment with a new technique works (the experiment is successful) the team’s effectiveness increases.  When an experiment “fails” their effectiveness dips for a bit but then rises back to where it was once they go back to their previous WoW.

Figure 2. Team effectiveness improves over time by experimenting with new WoW (click to enlarge).

So what can we do to improve on this? The linchpin is the very first step in the process of Figure 1, the identification of a technique to experiment with.  As you see in Figure 3, when we improve the likelihood that a technique will work in our situation then our effectiveness rises faster due to more successful experiments.  We call this guided continuous improvement.

Figure 3. Guided continuous improvement increases the chance of successful experiments (click to enlarge).

It’s a simple idea – with better process decisions we achieve better process outcomes, as you can see in Figure 4. There are three ways that you can do this:

1. Hire an experienced coach (and listen to them). Although it can be
   hard to find an experienced agile coach they do exist and if you’re lucky enough to have one then follow their guidance.
2. Apply the Disciplined Agile (DA) toolkit. There are several ways you can apply the DA toolkit to help you to make better process decisions.  Instead of prescribing the “best practices” you must adopt, DA instead advises you on process-related issues you need to think about and the options available to you.  Such issues include what you should consider when choosing a lifecycle, what you should consider when forming your team, and what you should consider when addressing changing stakeholder needs (to name a few important things). The DA toolkit then presents you with potential options and the trade-offs associated with those options.  This gives you an idea as to what techniques you might want to experiment with and what is likely to happen if you do, enabling you to make better process decisions. This site overviews these decisions, as you can see by following the previous links, and the book Choose Your WoW! summarizes the trade-offs for Disciplined Agile Delivery (DAD).
3. Both of the above. Good coaches have the humility to recognize that they don’t know everything, and will leverage the DA toolkit to help your team to make better decisions about new ways of working (WoW) to experiment with.

Figure 4. Team effectiveness improves at a quicker rate with guided continuous improvement (click to enlarge). 

Continuous improvement, evolving your WoW through experiments, is a proven way to achieve lasting process improvement. Lean practitioners have been doing this for decades and virtually every DevOps case study advises you to evolve your WoW this way. Guided continuous improvement takes it one step further and streamlines your experimentation efforts.

MORE INFORMATION

For more information about choosing and evolving your WoW, we humbly suggest that you consider reading our book Choose Your WoW! A Disciplined Agile Delivery Handbook for Optimizing Your Way of Working. If you want to succeed at enterprise agile you need choices, not prescriptions.

A fundamental philosophy of agile is that teams should own their own process, or as we like to say in Disciplined Agile (DA) teams should choose their way of working (WoW).  Of course this is easier said than done in practice. The challenge is that every team is unique and faces a unique situation – in other words, context counts. Furthermore, there are no “best practices,” rather, every practice has tradeoffs and works well in some situations and poorly in others. Worse yet, you really don’t know how well a technique will work for you until you actually try it out in your environment.  Given all of this, how can a team choose its WoW?

Since the 1980s the lean community has shown that an effective way to evolve your process is to do so as a series of small incremental improvements, a strategy called kaizen.  Numerous organizations have taken this approach over the years, and virtually every single DevOps success story is based on a multi-year kaizen-based continuous improvement strategy. Figure 1 depicts the workflow of implementing a single improvement, with Deming’s Plan-Do-Study-Act (PDSA) loop on the right-hand side to indicate the iterative nature of the overall process.

Figure 1. Running an experiment to evolve your WoW (click to enlarge).

Let’s explore each step one at a time:

1. Plan: Identify a potential improvement. The team identifies a technique, either a practice or strategy, that they believe will work for them. This may be something someone on the team has done before, something they’ve read about, or even an idea that they’ve identified on their own.  An important consideration is to try a technique where it is “safe to fail” at it – you shouldn’t be putting your team at risk by experimenting with a new WoW.  If an experiment is deemed to risky to try, attempt to  break it down into a series of smaller experiments that are less risky. It is also critical to identify the criteria against which you’re going to assess the effectiveness of the technique.  Ideally most of the criteria is quantifiable in nature, although don’t ignore qualitative measures too.
2. Do: Try out the new WoW (experiment). The team needs to see how well the technique works for them in their environment.  Do they have the skills to perform the technique? How well does it work given their technology platform?  How well does it work given their organization and team culture? The team needs to give the experiment sufficient time to determine how well it works in practice. This could several anywhere from several days to several months, although in most cases several weeks is sufficient.
3. Study: Assess effectiveness. After you’ve run the experiment you should assess how well it worked for you. This assessment should be against the criteria that you agreed to at the beginning.
4. Act: Adopt or abandon the new WoW.  Sometimes you’ll find that a new technique works incredibly well for you, and other times you’ll experience the other extreme and discover that a technique is an abysmal failure for you.  But in most cases there will be some aspects of the technique that work well for you and other aspects that do not (an indication that maybe you need to consider running a new experiment to identify a solution). Our advice is to adopt what works well for you and to abandon or better yet improve upon what doesn’t.
5. Act: Share learnings with others. DA teams are enterprise aware, recognizing that they are only one of many teams within your organization. So, when a team learns something about a technique the implication is that they should share it with others.  Strategies for doing so are addressed by the Evolve WoW process goal.

Figure 2 shows how your team’s effectiveness improves over time with a continuous improvement approach.  When you experiment with a new technique and it works out well for your team then your team effectiveness improves.  When an experiment “fails” your team effectiveness dips for a bit – the technique didn’t work well in your situation – but then you end the experiment and go back to your previous way of working (your team’s level of effectiveness goes back to where it was).

Figure 2. Experiment to evolve your WoW (click to enlarge).

When you keep at it, when you adopt a kaizen mindset, your team effectiveness increases over time as we see in Figure 3. The figure also shows that when you first adopt a continuous improvement strategy that your team effectiveness drops at first because you’re learning how to follow the continuous improvement process of Figure 1.  In many ways you begin this improvement journey by experimenting with this experimentation-based strategy.

Figure 3. Continuous improvement over time (click to enlarge).

Some organizations struggle with the idea of experimentation, likely because they still believe in the idea of “best practices” and often because they’re looking for an easy answer. They’re afraid to experiment because they might “fail,” not realizing that a failed experiment teaches you what doesn’t work for your team given your current situation. Running small, “safe to fail” experiments is absolutely critical for improving your WoW.

Where this blog overviewed the strategy of Continuous Improvement, in the next blog in this series we will see how better decisions lead to better outcomes via Guided Continuous Improvement.  Stay tuned!

More Information

For more information about choosing and evolving your WoW, we humbly suggest that you consider reading our book Choose Your WoW! A Disciplined Agile Delivery Handbook for Optimizing Your Way of Working. If you want to succeed at enterprise agile you need choices, not prescriptions.

The Disciplined Agile (DA) tool kit references hundreds of agile, lean, and in some cases “traditional” techniques, putting them into context. One such technique is database refactoring, adopted from the Agile Data (AD) method, which is critical for the Improve Quality process goal. Where a code refactoring is a simple change that improves the quality of code, a database refactoring is a simple change to your database that improves the quality of its design without changing the semantics in a practical manner. Database refactoring is a sophisticated practice, in fact I co-wrote a 400+ page book about it, Refactoring Databases: Evolutionary Database Design, with Pramod Sadalage in 2006. The book, which won a Jolt Productivity Award, provides a detailed description of the practice. But what if you’d like to start with something simpler before jumping into the details? This is where Essence comes in.

A few months ago a group of us – Roly Stimson of IJI, Nebojsa Trninic, Vladimir Savic, Ervin Varga, and myself – decided to experiment with the idea of essentializing DA. To focus our work we decided to start with a simpler problem, that of essentializing a subset of the techniques referenced by DA, in this case those of Agile Data. We then focused further on a minimal viable product (MVP), in this case the single practice of database refactoring.

We captured the practice using Essence Enterprise 365 from IJI, a screen shot from which is shown below. Essence is a language for describing software engineering methods and practices from Ivar Jacobson International (IJI). Essence was created as a part of Software Engineering Method and Theory (SEMAT) and approved by The Object Management Group as a standard in 2014. I’ve been working with SEMAT from its beginnings in 2009 and have known (and admired) Ivar since the late 1990s.

As you see in the screen shot we summarized database refactoring using the tool, describing the practice as a collection of cards. The tool can be used to develop process material, as we’ve done, and more importantly it can be used by teams to tailor and evolve their own process. Within the tool are dozens of practices, including ones from Scrum and XP, that a team can configure to reflect their own way of working (WoW).

In the following picture you see that these cards can also be printed out and used in a physical manner, often for process tailoring or training. In November I experimented using the cards to train a small group of people. We played a game where the group worked through how they would go about refactoring an existing database. They discussed each card, moving them around on the table until they had something that looked like the picture below. The cards provided a tactile way for them to explore database refactoring by thinking through how they would apply it in practice. IJI has identified a collection of teaching and process tailoring games using cards such as this.

An interesting result of this teaching experiment is that we discovered that we wanted more cards. The group wanted cards to explore automated database regression testing, agile data modelling, and continuous database integration in detail, and of course how all of these things fit together. So it looks like we have our work cut out for us.

We’d love to hear what you think about this effort. This blog has been a brief description of our work to essentialize the practice of database refactoring, but we have a lot more work ahead of us. A more detailed description will be presented in a forthcoming research paper that the team is working on. We also intend to continue with this essentialization effort to share some of the key DA techniques within Essence. Stay tuned!