Many people in the Agile community consider a 19year old document to be the definition of Agile. While it can still serve as inspiration, I believe something that happened at the midpoint of the PC era to be defining something that is supposed to adapt and change over time to be somewhat oxy-moronic. I am referring, of course to the Manifesto for Agile Software Development (MASD). There are several reasons that I do not believe this is an efficacious approach:
This does not mean Agile is dead. It just means we should use another meaning for Agile. I believe the best vision of Agile is Steve Denning’s Three laws:
This essentially means to have small teams working towards the benefit of the customer in a network of semi-autonomous manner.
To say complexity stops us from knowing if something is going to be an improvement is like saying walls stop us even when doors are present in the walls. This attitude about complexity has us ignore available doors to try. We may not always go in the right direction but we can figure it out from what we learn after passing through the door.
Preset approaches often lead to a lack of “fit for purpose”
This post was originally written 10/10/2009
In our book, Lean-Agile Software Development: Achieving Enterprise and Team Agility, we mention that the "inspect and adapt" is not the same thing as Plan-Do-Study-Act. Yes, they sound the same, but they are manifestations of different causality models. To fully understand the differences in inspect and adapt and PDSA, we must look at these causality models (which we will do shortly).
Scrum suggests that because we are working on non-deterministic systems, our own process should be a controlled black box. See The Scrum Papers, pg 58. We disagree. A simple example of this is driving a car. I think we'd all agree that driving is a non-deterministic system. When you leave your home for work you do not know exactly what will happen – who will cut you off and how you'll have to drive defensively, or perhaps who you will cut off in an attempt to make up some time. Yet, most of the time, you get to where you wanted to get to in about the time you expected to get there. The "process" you use – keeping a certain distance between the car in front of you and your car (except, of course, when you are really trying to speed up the idiot), driving on the right side of the road when in the US on two-way streets, pressing the gas to accelerate, pressing the brake to slow down, threatening the kids if they don't shut up, … While simply knowing the rules doesn't mean you know how to drive, knowing the rules helps you drive better.
In an earlier blog (Types of Processes by Don Reinertsen), I discussed how the degree of visibility of your process is a separate issue from the level of randomness of its output. A third, separate, issue is how much feedback you need to control things.
While we agree that software development is a non-deterministic process, we do not believe that there is no causality of the actions involved. We also believe that it is important to create visibility into the process (what we call transparency) and not just visibility into the results (I've stated this several times on user groups but will write an upcoming blog to focus on this shortly). This is a significant difference in flow based systems (e.g., Kanban) and Scrum.
At a cursory level, the project boards for both flow based systems and Scrum look the same. One can see work entering the system, different stages in which the work is in and when the work is done. We call this visibility – that is, we have visibility into the results of the team. Incredibly important, but insufficient in our minds.
As important is the answer to the question- how does the work flow from one end of the board to the other? Is it just up to individual member's decisions on when to work on things or is there a visible set of decisions at work? While no complete definition of Scrum exists, the aforementioned Scrum papers as well as many blogs and user-group comments from CSTs (certified scrum trainers, presumably the highest authorities on what the Scrum Alliance's stance on Scrum is) continuously state the supposition that a defined process is not a good idea (if it were even possible). In other words, most Scrum boards will show you stories waiting to be worked on, those in process (including varying states) and those completed but having a defined set of rules for how things go from one column to the other is not a part of Scrum. It is just left to the team members' judgment. Teams are supposed to pay attention to the effectiveness of their actions and adapt accordingly. You can, of course, add your own rules. Good thing to do in our mind if you are doing Scrum – but then you have a variant of it.
PDSA is a bit different from "inspect and adapt" in that it requires the team to consider how things are actually working. In other words, we don't want to just inspect and adapt, we want to understand, at least a little, about the causality of things. For example, agile teams often experience a backlog of tests for testers at the end of a sprint. Developers have done the coding but the testers don't quite seem to be able to keep up. A good Scrum team will try to figure out what actions they can do to help here. Perhaps they try to pair developers together with testers. Perhaps they decide to specify their tests before writing code. Both good things. But what they haven't done is to try to understand the dynamics of what is going on by creating a model of it – something they can then try different things against to see what works better. Given one tenet of Scrum is your process is black box, i.e., it shouldn't (can't) be defined, this is not a surprise.
Lean takes a scientific approach. It believes you can understand the affects that your actions have. Lean suggests that one should consider how they are working to be the best way they know how. In this regards, their method of working is an hypothesis – "this is the best way to do our work." We make improvements to how we work by suggesting new a new hypothesis and seeing what happens. That is, we see how our actions affected our results. In Kanban, we focus on managing work in process levels. Our process hypothesis typically includes a set of limits of different types of work plus service level agreements. We adjust these to maximize value delivered to the customer.
In our developer / tester disconnect example, we might also consider pairing developers and testers, but we should also see if we get any improvement. This is how the PDSA cycle works. Our "plan" is "pairing developers with testers." We are hypothesizing that this will be an improvement. Our "do" is trying this out for some time specified by the team. We then "check" to see what happened (validate or invalidate our hypothesis). And then we "act" accordingly – make a new model, way of working, etc. Note, in knowledge work a somewhat equivalent model - Look, Ask Model, Discuss, Act (LAMDA) – has been offered. This probably provides a better metaphor, but is essentially the same intent.
While Scrum treats the team's process as a black-box, Lean treats it as a transparent process which requires feedback to keep it under control (since we are working on a non-deterministic system). We call this characteristic transparency.
Let me illustrate the difference between inspect and adapt and PDSA in a non-software world example. I'm a beginning sailor (have been for 30 years ;) ). I'm not really that skilled, but I do know a few things. When I first started sailing they said to look at the little ties on the sail to see which way the wind is blowing from. As the ties blow around, you adjust the sails to meet where and how strong the wind is coming. This is a simple rule of sailing. Note how it is reactive.
Now, pretty soon, you notice how it'd be great if you could get some sense of what the wind is doing before it hits your sail. Unfortunately, you can't see wind. But you can see the affects of the wind. I remember being told to look at the waves. But I, at first, did not find that very useful. Eventually, it started making sense to me. The primary affect of wind on water is the size of the waves and direction of the waves. But these are due to the prevailing wind – what I needed was the more micro changes that were affecting my sailing. Abrupt wind changes do not change the general height or direction of the waves. However, these do set up ripples on the waves. I started wondering if I could see a pattern between how the wind changed and how the ripples appeared. In other words, if there were different ripple patterns approaching me, did that mean the wind was changing? I would make guesses in my mind and see what happened. I would then see how I could respond to this. Over time I learned to "see wind coming at me." I wasn't merely inspecting and adapting I was PDSAing. :) I had created a theory, tried it out and then used that new theory to improve my behavior.
Adoption and Where to Use Scrum
Scrum was designed for a cross-functional team to building new product incrementally. Using it everywhere follows the mantra “when all you have is a hammer, everything looks like a nail.”
Scrum ignores the potential adverse consequences (e.g., resistance) that forced adoption of a new, predetermined way of working will entail.
Scrum provides no guidance on where it should be used other than to solve complex problems and that people should want to improve things. Of course, nothing works when no one wants to improve things. The statement basically says if a team wants to improve then Scrum will work, implying that when it doesn’t work people weren’t committed enough..
“Scrum would work if people would use it” is circular reasoning. When it’s not enough people will have a tougher time using it and they will fail in their endeavors. But it may have more to do with the lack of fit for purpose than their commitment.
Lack of Guidance and Theory
Scrum intentionally provides no guidance or theory as to why it works. The idea is that it makes it simpler, you can't provide all the choices needed anyway and people will figure things out. Evidence, unfortunately, has shown that people don't figure things out and Disciplined Agile has demonstrated there are ways to provide choices without being complicated.
When Scrum is used outside of where its practices were designed for (cross-functional teams and incremental implantation is straightforward) it provides no guidance on how to achieve these.
Provides no guidance on where it should be used and ignores the issue of fit for purpose
Scrum provides no guidance on skills needed. The preposition that people will figure out what’s missing in scrum is invalided by reality.
The statement that Scrum is intentionally simple and incomplete should not imply that this conscious decision was a good one.
The implication by Scrum practitioners that incomplete makes it easier to understand ignores the possibility that there are ways to design approaches that are easy to understand yet provides information as needed. Simple to use does not imply incompleteness. In fact, the opposite is often true.
While it’s true that nothing is complete, that does not argue for extreme incompleteness.
Difficulty of Scrum Due to Lack of Guidance and Theory
Scrum is both easy to understand and easy to master. It’s using Scrum to master Agile that’s difficult.
The belief that Scrum is easy to understand but difficult to master comes from conflating Scrum with Agile. They are two different things. One is a way of being and doing (Agile) and the other is a framework for developing, delivering, and sustaining complex products.
When the immutable aspects of Scrum are not present, Scrum provides no guidance on how to create them.
The lack of choices in Scrum requires you to re-invent the wheel in many places.
The lack of theory often leads to Scrum But (stopping a Scrum practice in a way that keeps the impediment in place) because there is no way to determine is another known practice is appropriate..
Scrum provides no guidance in how to work in situations where cross-functionality or iterative development are difficult to achieve. Nor does it provide any guidance on how to achieve these.
Because Scrum provides no theory as to why it works (e.g., Flow, Lean, ToC) practitioners doing Scrum must keep trying to do it – whether it is the correct course of action.
Scrum requires its roles, events, artifacts and rules to be immutable because there is no theory presented to explain how to change them. The mindset underneath this is just do Scrum as it is and things will work. But this conflates Scrum with the result (Agile) you want to achieve.
How Scrum Can Become a Framework Prison
Ivar Jacobson wrote an article called Escaping Method Prison where he talks about the methods we use become our prisons. I would suggest that when our method is built around a framework, a prison with a moat around it is created. I call this a "framework prison." A similar article Framework Tunnel Vision predates this. I described how frameworks and methods tend to have people overly focus on certain practices. The way to escape this overly narrow focus is to enable one's approach to expand to include any useful concept. While many people claim that this will make things cumbersome or overly complex, I assert that that's only if you don't know how to design the approach properly. More information is not necessarily more complexity. Wikipedia is an example of this.
While "Method Prison" is to be avoided, "Framework Prison" is a bit harder. The reason is that frameworks are built on top of key values, concepts, practices, roles, etc. These cannot be removed and still have the framework be intact. While you may add things to the framework you can't usually take things out without breaking it. Scrum explicitly states that its roles, events, artifacts and rules are immutable. SAFe may not say that but few options exist to change its fundamental concepts without breaking it.
Because Scrum is based on values and practices and provides no theory as to why it works, those certified in Scrum Guide based Scrum feel compelled to follow these values and practices. However, changing people’s values is difficult. If the practices are not applicable or not "fit for purpose" there will also be problems. When someone leading a Scrum adoption knows only Scrum two problems occur. First, they may not see more useful alternatives. The second is more insidious however. If they were brought in to "adopt Scrum" they may feel some fear in trying to do something else - both outside of their range of certification and not what they were brought in for.
Changing Scrum is risky because there is no theory to tell you if the change you make is a good one. It is psychologically hard to make the change as well if you are only trained in Scrum since you have to go outside of the definition of what Scrum is.
The fact that it is risky to customize Scrum speaks to its poor design/mindset. When one is trained only in Scrum it is often difficult to see better ways to achieve the goals that people adopted Scrum for.