There is a lot of talk about complexity now in the Agile community. I am not a big fan of sense-making frameworks that talk about simple, chaos, complicated and complex.  But I wrote this post on linkedin (including it here) to let us know when you are in a complex system.

Go through the questions one by one. Give yourself 2 points for each question answered "definitely" and one point when answered "to some extent." At the end we'll total up your score and tell you if you're in a complex system.

1. are you doing knowledge work?
2. are there more than 20 people involved?
3. do you have to interact with people outside of your group for which you have no control?
4. are there many dependencies between the different groups involved?
5. is much of what's happening not visible to people?
6. can small misunderstandings cause big challenges?
7. are you doing something never done before?
8. are you trying to learn how to do what you're doing better?
9. are people's livelihoods at stake based on the success of what they are doing?
10. is it impossible to keep everything that's going on in your head?

Now, total up your answers. If your score is 2 or more you're in a complex system.

The question isn't if you're in a complex system, the question is what are you going to do about it? How are you going to navigate it?

-----

To learn more about how to deal with complexity, check out Dealing With Complexity by Creating a Bias for Simplicty

Scrum as a Double-Edged Sword

One of the attractions of Scrum is its simple definition and ability to start a team quickly. But this has proven to be a double edged sword. While one of Scrum’s creators, Ken Schwaber, implores “Scrum is simple, just use it as is” many teams find they can’t get the value out of Scrum that they had thought they would. Teams struggling with Scrum abound. So much so that they’ve been given names – “dark Scrum” and “Scrummerfall.” The reasons for this is Scrum’s simple start is, unfortunately, simplistic. Scrum is based on a set of values and empiricism. This enables it to provide a how and who. It’s lack of theory makes it unable to provide the required what and why. Empiricism alone is insufficient in that it doesn’t provide the forward looking ability to predict what changes may be useful that theory does. Scrum’s claim of being based on Lean-Thinking is unfounded*. This leaves Scrum practitioners without the knowledge that is necessary to fill in the gaps and enable Scrum Guide Scrum to be modified to avoid the challenges so many people face with using it.

Applying Lean theory will almost certainly take people away from the immutable aspects of Scrum, so they will no longer be doing Scrum. This tends to subtly create a limitation on thinking since many Scrum practitioners were only trained in Scrum (especially those certified as ScrumMasters) so there is a natural reluctance to look outside of Scrum's allowed practices. While this limitation can be a good thing when people don't understand Lean, it creates a situation where practices that are not ideal for the team are being followed.

The concept of “Scrum” originates in the New New Product Development Game by Hirotaka Takeuchi and Ikujiro Nonaka, published in 1986. This entails:

1. Having a team that understands or can learn what is needed

2. Setting up a process to achieve quick feedback on what is being built

These two concepts provide the rationale for the two iconic aspects of Scrum – the self-organizing, cross-functional team and using sprints, or iterations in the generic Agile vernacular. This combination enables the team to pivot after working towards an interim solution. The term Scrum comes from the sport of rugby. A scrum in rugby is a way of restarting play. Scrum’s daily scrum is done to restart the day, to see where impediments are and how to get past them. Scrum itself is based on the regular “restart” of what’s being built. Teams sprint in creating an aspect of the product and then stop, reflect, and restart for the next sprint.

The challenge is that Scrum’s lack of theory as to what works in knowledge work requires it to define many prescribed practices which are immutable since no theory is provided to explain what other practices might work better in certain situations. While these will often work in situations where Scrum was originally designed (new product development by a cross-functional team) Scrum is now more often used in places outside of where it was designed for. Fortunately, just a little Lean-thinking can provide this necessary theory.

A Quick Introduction to Lean-Thinking

The essence of Lean-thinking is learning quickly. Taichi Ohno, the creator of the Toyota Production System from which Lean originated, was wont to say to his managers “if I come back here in 30 days and you’re still doing the same thing then you haven’t done your job.” Lean is based on the following:

• Systems thinking. That is, looking at the whole and the relationships between the components in the system and recognizing that most of the behavior, and errors, that take place are the result of the system.

• Just-in-time. This means avoiding doing things early since the delay caused by doing so creates waste. This is typically implemented in a pull system which also decouples events from each other to avoid a cascade of timing errors from occurring.

• Fix common cause errors are soon as they are detected (stop the line). Systems theory tells us that almost all of the errors are due to the system, so when an error occurs we must fix the system.

While Scrum does partially follow some aspects of Lean-thinking, its equally violates it. In particular, Scrum’s foundation on batches (sprints) of work is the anti-thesis of Lean. This is particularly true in retrospectives taking place at the end of a sprint instead of on a more ongoing basis. Lean is about continuous learning, not learning after a batch of work. This quote by Edwards Deming, sums this up nicely “Experience teaches nothing. In fact there is no experience to record without theory… Without theory there is no learning… And that is their downfall." People copy examples and then they wonder what is the trouble. They look at examples and without theory they learn nothing.” Without a theory as to how to do knowledge, work, Scrum is doomed to remain in its own echo chamber. This is perhaps the biggest manner in which Scrum is inconsistent with Lean-thinking - Lean does not put any boundaries on what can be done - it is completely pragmatic and is about achieving results for the situation at hand, not by using pre-defined practices.

Taking a Disciplined Approach to Scrum

If we reformulate a team approach to creating new products based on the New New Product Development game but use Lean thinking to provide the why to Scrum’s prescribed how we can understand the what as well. That is, instead of starting with how to do new product development, let’s look at what’s required by understanding the theory underneath the work. This provides us with the why needed.

When we have the what, how, why and who, we are no longer constrained to the domain discussed in The New New Product Development game. That is, we can now figure out how to create new hows in the new context. This is the essence of Disciplined Agile’s choose your way of working. Don’t start with how but look at what you are trying to accomplish, understand the why and decide on the how and the who within your own context. 

The following tables describe what a Disciplined Agile approach to defining a team process would look like. It’s organized into three main sections: what (the intention we want to accomplish), how (a suggested practice or practices), and the why for both the whats and hows presented. This combination enables us to start with a way of working and then to modify it while ensuring we still accomplish what’s needed. The why column is based mostly on key Lean principles.

The table has two columns for the hows, one for an iteration based approach and one for a flow based approach. There is an ongoing debate as to which is easier to start with and our belief is that not only is there no correct answer, but each approach can be improved by attending to the other. Iterations provide more structure and can therefore be a better approach to teams. However, flow is an inherently more efficient method. Aspects of flow can often be incorporated into an iterative model and are shown when appropriate.

It is important to note that the collection of hows for the iteration approach is not Scrum Guide Scrum but rather an iterative model based on the whats and whys of knowledge work. It therefore includes practices that both substitute for and add to Scrum’s prescribed practices. One could say it’s a combination of ScrumButs that work, and ScrumAnds that extend.

The numbers in the what column refer to the factors for effective value streams we've found to be useful. They are summarized in the following table.

Note that this is a work in process. This is the first part of an article that will be expanded to present a guide to team agility that is about 15-20 pages long, but designed without boundaries, options to choose from and the thought process to make good choices.

If you want to learn more about having an Agile team approach that works for you I suggest looking at Disciplined Agile's Scrum Master (DASM) Certification. Or, if you are already familiar with Scrum and want to learn how Scrum teams can work together, check out the Disciplined Agile's Senior Scrum Master (DASSM) Certification.

* In ’07 Al Shalloway, on the Yahoo Scrum Development group, suggested that Scrum was a partial manifestation of Lean. Ken Schwaber vehemently disagreed, expelled Mr. Shalloway

The “what”, “how”, “why” and “who” aspects of knowledge work need to be addressed. Although Simon Sinek’s “Start With Why” has captured the imagination of many agilists, virtually all Agile frameworks start with “what”, some not even mentioning the “why.”

Starting with why provides both an understanding of goals, provides respect for the team and creates the possibility of discovering a better way to work than any prescribed approach.

Frameworks typically start with how because that’s what frameworks are about – how to accomplish something. It’s important therefore to be clear about the why and what when using them, especially because they often won’t tell you that.

Let’s take a simple example – iterations (a how) in Scrum. There are other ways to achieve what they achieve. The “what” iteration is a “how” to is:

• Limit work in process
• Work on small items
• Get feedback no longer than the timebox
• Provide a cadence for different roles to work together & for learning

These are all good things. But there are many other ways to achieve them. Understanding the why of these what’s is useful. Mostly to remove delays in workflow &feedback, both of which cause extra work.

Understanding the what & why provides you the ability to discover better "how"s for your situation.

Scrum org defines “ScrumBut” as modifying Scrum in a way that doesn’t solve a problem but makes it look solved. Scrum requires following its practices until you learn how to do them. Unfortunately, these bad ScrumButs often result when a practice is not fit for purpose and/or the team can’t figure out how to apply it,
 
DA takes a different approach. DA, of course, doesn't suggest just abandoning a practice. It recognizes that no practice is universal and that there are many options to solve a challenge.  It helps teams find a better one with a pragmatic mindset, a toolkit to provide options & a little theory to help in their selection.
 
DA's toolkit, organized by goals, lays out the decisions required & options to achieve them. It doesn’t take a lot of theory to make the selection. Are delays in workflow being removed? Are handoffs being reduced? Is there quick feedback amongst all interested parties? Moves that improve these are good.

This combination of pragmatism, options & theory enable teams to select fit for purpose solutions to their specific challenges. Just as important, teams learn to solve their own challenges and not be required to follow someone else's solutions.

In defense of root cause analysis

First, let me acknowledge that root cause analysis (RCA) is often misused. It is not intended to lay blame. This use is rooted in a lack of understanding of systems thinking as espoused by Edwards Deming. Deming proposed the theory that most errors are due to the system while some are due to special causes that happen infrequently

Doing RCA on special causes is counter-productive. However, RCA on causes due to the system are worthwhile. Lean management is based on this idea - improve the system, stop blaming people

A 2nd misuse of RCA is actually believing there is one cause & that the point of RCA is to get to it. This may work in manufacturing where limiting variation is a goal. In knowledge work there are too many variables to limit. While RCA may get you to a root cause, ii is more likely to provide a deeper understanding while leading to several potential root causes

The challenge I have w/those dismissing RCA off the cuff is they bring up examples of it being misused while ignoring cases where it has proven useful. Go here to see an example where by using "5-whys" an organization increased the productivity of a 100 person dev team by 20% while also greatly improving the quality of their product in a matter of 3-4 hours