Continuous Improvement Through Experimentation

 

A fundamental philosophy of agile is that teams should own their own process, or as we like to say in Disciplined Agile (DA) teams should choose their way of working (WoW).  Of course this is easier said than done in practice. The challenge is that every team is unique and faces a unique situation – in other words, context counts. Furthermore, there are no “best practices,” rather, every practice has tradeoffs and works well in some situations and poorly in others. Worse yet, you really don’t know how well a technique will work for you until you actually try it out in your environment.  Given all of this, how can a team choose its WoW?

Since the 1980s the lean community has shown that an effective way to evolve your process is to do so as a series of small incremental improvements, a strategy called kaizen.  Numerous organizations have taken this approach over the years, and virtually every single DevOps success story is based on a multi-year kaizen-based continuous improvement strategy. Figure 1 depicts the workflow of implementing a single improvement, with Deming’s Plan-Do-Study-Act (PDSA) loop on the right-hand side to indicate the iterative nature of the overall process.

Figure 1. Running an experiment to evolve your WoW (click to enlarge).

Let’s explore each step one at a time:

1. Plan: Identify a potential improvement. The team identifies a technique, either a practice or strategy, that they believe will work for them. This may be something someone on the team has done before, something they’ve read about, or even an idea that they’ve identified on their own.  An important consideration is to try a technique where it is “safe to fail” at it – you shouldn’t be putting your team at risk by experimenting with a new WoW.  If an experiment is deemed to risky to try, attempt to  break it down into a series of smaller experiments that are less risky. It is also critical to identify the criteria against which you’re going to assess the effectiveness of the technique.  Ideally most of the criteria is quantifiable in nature, although don’t ignore qualitative measures too.
2. Do: Try out the new WoW (experiment). The team needs to see how well the technique works for them in their environment.  Do they have the skills to perform the technique? How well does it work given their technology platform?  How well does it work given their organization and team culture? The team needs to give the experiment sufficient time to determine how well it works in practice. This could several anywhere from several days to several months, although in most cases several weeks is sufficient.
3. Study: Assess effectiveness. After you’ve run the experiment you should assess how well it worked for you. This assessment should be against the criteria that you agreed to at the beginning.
4. Act: Adopt or abandon the new WoW.  Sometimes you’ll find that a new technique works incredibly well for you, and other times you’ll experience the other extreme and discover that a technique is an abysmal failure for you.  But in most cases there will be some aspects of the technique that work well for you and other aspects that do not (an indication that maybe you need to consider running a new experiment to identify a solution). Our advice is to adopt what works well for you and to abandon or better yet improve upon what doesn’t.
5. Act: Share learnings with others. DA teams are enterprise aware, recognizing that they are only one of many teams within your organization. So, when a team learns something about a technique the implication is that they should share it with others.  Strategies for doing so are addressed by the Evolve WoW process goal.

Figure 2 shows how your team’s effectiveness improves over time with a continuous improvement approach.  When you experiment with a new technique and it works out well for your team then your team effectiveness improves.  When an experiment “fails” your team effectiveness dips for a bit – the technique didn’t work well in your situation – but then you end the experiment and go back to your previous way of working (your team’s level of effectiveness goes back to where it was).

Figure 2. Experiment to evolve your WoW (click to enlarge).

When you keep at it, when you adopt a kaizen mindset, your team effectiveness increases over time as we see in Figure 3. The figure also shows that when you first adopt a continuous improvement strategy that your team effectiveness drops at first because you’re learning how to follow the continuous improvement process of Figure 1.  In many ways you begin this improvement journey by experimenting with this experimentation-based strategy.

Figure 3. Continuous improvement over time (click to enlarge).

Some organizations struggle with the idea of experimentation, likely because they still believe in the idea of “best practices” and often because they’re looking for an easy answer. They’re afraid to experiment because they might “fail,” not realizing that a failed experiment teaches you what doesn’t work for your team given your current situation. Running small, “safe to fail” experiments is absolutely critical for improving your WoW.

Where this blog overviewed the strategy of Continuous Improvement, in the next blog in this series we will see how better decisions lead to better outcomes via Guided Continuous Improvement.  Stay tuned!

More Information

For more information about choosing and evolving your WoW, we humbly suggest that you consider reading our book Choose Your WoW! A Disciplined Agile Delivery Handbook for Optimizing Your Way of Working. If you want to succeed at enterprise agile you need choices, not prescriptions.