Disciplined Agile Applied

As a manager, the quality of the data available to you has a direct impact on your ability to make effective decisions. I ran an informal survey exploring the state of data quality within organizations from April 11 to May 22, 2022.  The survey received 66 responses in total. This blog posting shares the key findings of that survey.

Figure 1 explores the perceived importance of data within organizations.  This year 95% of respondents indicated that data was considered to be an important asset within their organizations, which is consistent with previous studies that I have run in the past.  However, only 54% indicated that they were measuring data quality, which tells me that in many organizations “data is an asset” is merely rhetoric. 

Figure 1. How important is data to your organization?

Data Technical Debt

The survey explored issues surrounding data technical debt (DTD), which is a measure of level of data quality (DQ) problems within a data source. Figure 2 summarizes the results of a question that explored the quality of the most recently accessed data source by the respondent. Only 42% of respondents indicated that the data quality was high or very high, and 19% indicated that the data quality was low.  Clearly room for improvement.

Figure 2. Quality of production data.

Figure 3 explores whether DTD is taken on intentionally, which is a management decision, with only 36% of respondents providing positive answers.  Once again, this is an indication that there is significant room for improvement in many organizations.

Figure 3. Is data technical debt taken on intentionally?

Addressing Data Technical Debt

I also explored whether organizations were addressing DTD effectively.  Figure 4 summarizes the result of the question that explored whether organizations had a DTD strategy in place.  Figure 5 summarizes the results of a question about the adoption rate of traditional data quality strategies and Figure 6 the adoption rate of agile data quality strategies.  In general agile quality strategies are more effective in practice than traditional strategies.

Figure 4. Do you have a strategy to address data technical debt?

Figure 5. Adoption rate of traditional data quality techniques.

Figure 6. Adoption rates of agile data quality techniques.

As with many issues within the agile community, there is a bit of confusion around the pragmatic application of the Definition of Ready (DoR) and Definition of Done (DoD). In this blog I answer frequently asked questions (FAQs) around these two techniques.  These questions are:

• What are the Definition of Ready (DoR) and Definition of Done (DoD)?
• Why do we want a DoR?
• Why do we want a DoD?
• Can we define a standard DoR/DoD?
• Must a team have a DoR or a DoD? Both?
• What is the relationship between DoR and DoD?
• What must a DoD minimally cover?
• Do we need to negotiate our DoR and DoD?
• What if we can't come to an agreement?
• What happens when our DoR or DoD breaks?
• What happens when a team produces more than one thing, or more than one version of a thing?
• Are DoR and DoD quality gates?
• Aren't quality gates non-agile?
• How should we document DoR/DoD criteria?
• How should we implement the check/inspection process?
• Does DoR/DoD apply outside of software?
• Aren't there similar concepts in the software world?
• How does DoR/DoD relate to team working agreements?
• Why is this so complicated?
• Why is there a raging debate about DoR and DoD?
• What does DA advise?

What are the Definition of Ready (DoR) and Definition of Done (DoD)?

Figure 1 summarizes three key concepts that I will use throughout this blog:

• Definition of Ready (DoR). The quality criteria which an input must meet or exceed to be considered acceptable by the team.  The team reserves the right to reject any inputs that do not meet the DoR.
• Definition of Done (DoD). The quality criteria that a team promises that their outputs will meet or exceed. The basic idea is that something does not leave the team until it meets their DoD.
• Disciplined Agile (DA) team. DA teams are semi-autonomous in that they find that they sometimes need to collaborate with others outside of the team to achieve their goals - the outputs of others are inputs into a DA team.  DA teams are self-organizing in that they decide how they are going to work together to fulfill their desired outcomes, producing outputs for their customers (which may be internal or external to the organization).

Figure 1. Defining DoR and DoD.

Why do we want a Definition of Ready (DoR)?

As you can see in Figure 1, your team's DoR helps to protect you from poor quality inputs. When your inputs are insufficient they put your work at risk, forcing you to either rework the inputs to get them up to the level that you need or to cut the quality of your own work. By reducing rework, or in some cases hand-backs from the customer to the supplier, you help to optimize the flow between the teams.

For example, when you're making a salad, you very likely have a mental DoR for the quality of the ingredients.  You want fresh lettuce, fresh toppings, a nice dressing, and so on.  But what if the lettuce isn't sufficiently fresh?  Perhaps you cut off some of the spoiled leaves, forcing you to do extra work to get it to the point where it's usable.  Or perhaps the lettuce isn't quite as crisp as you would prefer, reducing the overall quality of what you produce and reducing your enjoyment of it.

Why do we want a Definition of Done (DoD)?

As you can see in Figure 1, your team's DoD helps to protect your customers from shoddy work. It is effectively an agreement between you and your downstream customers that puts a lower limit on the quality of your outputs. Your DoD helps to ensure the quality of your outputs, which in turn improves the predictability of your team in that your outputs are consistent and it helps to optimize the flow between teams due to reduction in hand-backs.

Once again, consider making a salad. If you produce a salad that is below your usual level of quality you will likely disappoint whomever you made it for.  If the salad is for you perhaps you can live with that.  If the salad is for your children perhaps you will have to endure their complaints, or even risk that they won't eat it at all.  If the salad is for a customer in a restaurant, you risk them sending it back forcing you to redo your work or they may choose to never frequent your restaurant again.

Can we define a standard DoR/DoD?

Sort of.

The challenge that we face is that our teams consume and produce a multitude of things, from tangible products to intangible concepts. And teams are unique, using those things in different ways.

For example, salads are the only item in my cooking repertoire.  I can make a wide variety of entrees, including spaghetti, grilled stakes, soup, and many more.  I can even make a range of deserts and drinks (I was a bartender at one point during university). Clearly the DoD for a salad, for a T-bone steak, for an apple pie, and for a margarita will vary.  There will also be similarities too, such make something delicious, make it look interesting, and don't poison people.

The implication is that any DoD or DoR standard will need to focus either on commonalities between the things that you produce or consume or be a checklist of potential issues to consider when formulating your DoR/DoD.

Must a team have a DoR or a DoD? Both?

In DA we believe that teams should choose their own way of working (WoW), including whether or not they want to adopt DoR and DoD as techniques. Because it's your choice, in the strictest sense of the question, the answer is no. DA isn't prescriptive that way, but as you learned above it's a very good idea to have a DoR and a DoD.

How do DoR and DoD relate to one another?

The outputs of one team are the inputs to other teams. These "teams" may be external to your organization.  For example, the lettuce that I used to make a salad came from a grocery store, which in turn came from a vegetable distributor, which came from a farm. The salad itself may be input into someone's overall meal.

Figure 3 summarizes the relationship between DoD and DoR, where the upstream team is the supplier to the downstream team, their customer. There are three situations to consider:

1. The supplier DoD is insufficient (DoD < DoR). In this case the quality of the work being produced by the upstream team does not meet the needs of the downstream team.  Either they will need to negotiate respective changes to their DoD and DoRs to get them aligned, or the downstream team will need to find a new supplier that meets their needs. 
2. The supplier DoD aligns with the customer DoR (DoD = DoR). The supplier perfectly meets the needs of the customer, in other words the output of the supplier meets the minimum quality criteria of their customer.
3. The supplier DoD exceeds the customer DoR (DoD > DoR). On the one hand this is potentially wasteful because the suppler is doing more than they need to do, they are effectively injecting waste though over producing (gold-plating). However, on the other hand, this provides the opportunity to delight your customers because you've exceeded their expectations.  

Figure 2. Examining the relationship between DoD and DoR.

What must a DoD minimally cover?

Sometimes your team produces outputs that are used by different teams in different ways, and each of those teams has their own DoR. The implication is that minimally your DoD is the union of your customer's DoRs for what you produce for them. What I mean by this is that your DoD must cover all of the criteria for all DoRs for downstream teams.

Consider the example in Figure 3. The output of the upstream team is being used by three different downstream teams. The DoR for Downstream Team #1 has two criteria, A and B. Similarly, Team #2 has criteria B and E and Team #3 A C and D. The implication is that the DoD for the Upstream Team must minimally address the combination of the three downstream teams, in this case A, B, C, D, and E.  

Figure 3. Your DoD is the union of the DoRs of your customers.

Sometimes the minimum is only a start when it comes to quality. The minimum will hopefully satisfy your customers, but it might not delight them.  The challenge is that if another supplier comes along with a better offering, perhaps in this case A' that extends A to provide higher quality, then you risk losing your existing customers to that new supplier because they've gone above and beyond your offer.

Another strategy, described below, is to have multiple DoDs, one for each product version.

Do we need to negotiate our DoR and DoD?

It depends.

My gut reaction is to say yes, because that would be the smart thing to do. But you're not always in a situation where negotiation is possible.  Consider the following scenarios.

1. The supplier is an external organization. You may be able to negotiate, particularly if you're a valued customer, or at least make suggestions.  You may also ask for a high-quality option from them. Or you may choose to find another supplier.  For example, if I'm at the market buying lettuce for my salad and I don't like the quality of what I see, I can ask someone there if they have anything fresher in the back, I could look at another type of lettuce or leafy green such as spinach, or I could go to another market.
2. The customer is an external organization. This is the reverse of the previous situation.  Are you willing to act on the requests and feedback of your customers?  
3. The two teams are in the same organization. Are we willing and able to negotiate with one another? Are we allowed to do so?  Does the supplier in this situation have the flexibility to act on suggestions and thereby evolve what they produce?  Does the customer in this situation have the flexibility to evolve what they'll accept?

What if we can't come to an agreement?

It depends.

This is an issue when:

1. You're the customer of a single-source supplier. In the short term that's a problem because you don't have the option to go elsewhere.  In the long term you may be able to motivate another organization to get into this line of business, or perhaps even do so yourself. 
2. You're the single-source supplier. Although you may feel safe because you've "locked in" your customers, this is a risky long-term strategy for you.  As soon as another option becomes available, particularly if it's seen as a better one, you risk losing customers. Organizations that don't focus on delighting their customers may soon find themselves in trouble.
3. The supplier and customer are forced to work together. Sometimes you're told by senior leadership that "you two groups need to work together, figure it out." This can be tough when your history of working together is poor, when the fit between your teams isn't very good, or when there is little apparent value in doing so. In this case, negotiating your DoR and DoD respectively will be an important part of "figuring it out."

What happens when our DoR or DoD breaks?

Learn from the breakage.

Your DoR/DoD breaks when it is no longer sufficient.  Perhaps the upstream supplier changes what they produce without first negotiating with you.  Perhaps the downstream customer evolves their criteria for what they require. Regardless, you should treat this situation as a learning opportunity which very often leads to improvement.

What happens when a team produces more than one thing, or more than one version of a thing?

If they are different offerings, then you need a DoD for each one.

For example, we should have a DoD for the salad that we produce. If we also produced chicken stir fry, then there would be a DoD for that too.  There may be overlap in the DoDs, for example we may have criteria around not poisoning people with our cooking, but there would also be differences between them because salads and chicken stir fries are very different things.

Similarly, if you have more than one version of something than you need a DoD for each version. Earlier we talked about how a DoD for a given output needed to encompass all the DoRs of downstream customers.  Another option would be to refactor your output into different versions, one for each customers, so as to meet their exact needs.  This increases your overhead, even if you have a clever configuration strategy you still need to manage the configurations, but has the advantage of being able to offer customized offerings to customers.

Figure 4. Versioning your DoD to meet specific customer needs.

Are DoR and DoD quality gates?

Yes.  

To be more accurate, they are a form or type of quality gate, the implementation if which is determined by the needs of the situation that you face. Context counts.

Aren't quality gates non-agile?

This question deserves a multi-part answer:

• Why do you even care about this?  Your true goal should be to be as effective as possible, not to be agile.
• It depends on whether your situation requires some form of quality gate.
• It depends on how you implement the quality gates that you do need. 

Once again, consider the DoD for the salad:

• If it's just for me, my "DoD inspection effort" will be non-existent because I will have made something that I like in the first place.  
• If I'm making it for my daughter I've got a pretty good idea about what she'll eat.  At most I'll give the salad one last quick look before putting it in front of her.
• If I'm a cook in a restaurant I've likely been coached on how salads are made at there so have a pretty good understanding on what to do, and worst case I would reach out to a colleague for advice. But that's not enough. The waiter picking up the food to bring to the customers will also look it over before bringing the salad out, and will pass it back to me if they feel I haven't done it well enough.
• If I'm making it for a fast-food restaurant then I've likely been trained in exactly how they want salads made - one of their competitive advantages is consistency of product throughout their chain.  There's likely a detailed checklist for exactly what ingredients to use, in what order, and how much to use. There's the official salad packaging to use, and there's likely defined time constraints that I need to conform to.  All of this will be well documented. The manager is likely keeping an eye on what I'm doing, particularly when I'm new to that task, and there may even be some sort of artificial intelligence (AI) system that is visually monitoring the food preparation to ensure that the salad meets the DoD.

Context counts.  The implementation of your DoD, and similarly your DoR, varies by the situation that you face.

How should we document the DoR/DoD?

I always recommend an agile approach to documentation:

• Don't write detailed prose when a list of concise points will do.
• Don't list concise points when a philosophy or principle will do.
• Don't set a philosophy/principle when automation will do. 

How should we implement the DoR/DoD check/inspection process?

The way that you implement the quality gate check/inspection for DoR/DoD depends on your situation.  You have several options available to you:

• Formal review. The inspection may be a manual and potentially onerous process, involving one or more people explicitly inspecting our work. In the restaurant, this could be someone standing there with a clipboard to rigorously check the salad. Formal reviews certainly leave a lot of room for improvement, but may be the best that you can do at the present movement.
• Informal review. In the restaurant, this was a waiter quickly looking over the salad before taking it out to the customer.
• Occasional, "spot" inspections. Every so often the product is checked for quality.  For example, in the restaurant the head cook may come by to look at what I'm doing when they have a moment to ensure that my work is of sufficient quality.
• Automation. Frictionless, fully automated checking, is becoming very common in the software and manufacturing realms. For our salad, the AI monitoring system that visually inspected the salad before sending it out to a customer.  This is effectively frictionless (except of course when my salad fails inspection).  
• Continuous inspection through the creation process. The inspection may effectively not need to happen in simple situations, such as when I made the salad for myself, because the checking against the DoD is happening during the implementation effort itself.  

And of course I could have walked through a similar example for applying a DoR on the ingredients going into the salad.

Does DoR/DoD apply outside of software?

Yes. DoR and DoD are fundamental concepts that apply to a wide range of products, both tangible and intangible, and services. 

Also, have I not belabored the salad example enough?

Aren't there similar concepts in the software world?

Yes.  In fact, this blog was originally motivated by a conversation that I had with Al Shalloway where I observed the relationship between supplier DoD and customer DoR was similar to that of superclasses and subclasses via the Liskov Substitution Principle (LSP). According to LSP a subclass should be more specialized than a superclass, similarly the customer DoR should be equal to or a subset of the supplier DoD.

There is also the concept of design by contract where you define the interfaces of software components, the contracts so that others know how to interact with those components. The implication is that you don't need to know the implementation details of the component, you just need to know what it does and how to interface to it.

How does DoR/DoD relate to team working agreements?

Your DoR/DoD definitions are an important part of your external working agreement.

Remember that there are two aspects of your team working agreement - the internal working agreement between team members that defines the ground rules for how you will work together, and the external working agreement that defines how your team will collaborate with others. 

As per the previous question, this is effectively design by contract for teams.

Why is this so complicated?

Your implementation of DoR/DoD only needs to be as complex as your situation warrants, as you saw in the examples above. The overall concept may seem complicated right now because I'm purposefully exploring all of the issues surrounding these two techniques. But as I discussed above, the way that you implement DoR and DoD is completely up to you. My advice is to keep this as simple as you possibly can.

Why is there a raging debate about DoR and DoD?

There are several reasons why people are still debating these concepts after all these years:

1. People like to argue.  Nuff' said.
2. Literalist thinking. Many people, particularly in the IT world where agile comes from, tend to think in literalist, or "black and white," terms.  As a result of literalist thinking, these people often struggle to recognize what they are actually doing in practice because it doesn't fit nicely into their desired categories.  For example, you'll often hear people claim that because they haven't written down a DoR or a DoD that they don't have one.  Yet in my salad example, you clearly have a DoR concerning the quality of the ingredients that you're willing to use even though it isn't written down.  You also have an unwritten DoD in that you're unlikely to serve a poor quality salad to your family, or in the case of a restaurant to your customers. Luckily you can typically reason with literalists, walking them through a few examples to help them understand that there are other ways of thinking about these concepts.
3. People confuse automated processes, or continuous inspection, with non-existence. Automation can effectively remove the friction of the inspection process to zero. Even though your team is no longer investing effort to check against the DoR/DoD criteria they are in fact still being validated against. Just because you've built the inspection effort into your overall development process, perhaps via automation or perhaps via adopting the habit of taking pride in your work, doesn't mean that a DoR or DoD doesn't exist - it just isn't explicit.
4. Purist thinking. In some cases the problem is with purist thinking that is based on an unrealistic vision of agile, such as the idea that quality gates aren't agile.  As you read earlier, that's clearly nonsense. Purist thinking often stems from a combination of literalist thinking and a narrow range of experiences applying agile in practice. One of the Disciplined Agile principles is Be Pragmatic, so we prefer to understand the advantages and disadvantages of a technique and then provide advice for how to apply them effectively in practice when appropriate.  
5. The "agile teams are autonomous" myth. Some agilists, particularly purists, don't like diagrams such as Figure 2 and Figure 3 that explicitly show how teams collaborate with one another. Collaboration between teams is an incredibly common occurrence. Yes, there is a desire to form teams that are cross-functional, have sufficient resources, and have the required skills to get the job done and that's a wonderful vision.  Now let's be be pragmatic.  Your team very likely needed to work with finance to get funding, it very likely was initiated via some sort of portfolio management effort, you're very likely being governed, you may need to get advice from your security experts to ensure your work meets your organizational standards, you may need to work with your data team to access existing data sources within your organization, you may need to interact with your support/help desk team when your customers run into issues, and so on.  At best your team will be semi-autonomous, having periods where you're autonomously but other periods where you're collaborating with others. DA teams are enterprise aware and explicitly choose to work with other groups in your organization so as to be more effective.

What does Disciplined Agile (DA) advise?

As always, look to the DA mindset for guidance:

1. Context counts.  Have a definition of ready (DoR) and a definition of done (DoD) that reflect the situation that you face. The salad DoR/DoD varied significantly depending on whether I was making it for myself, my family, or customers of a restaurant.
2. Be pragmatic. Your implementation of DoR/DoD should be just barely good enough (JBGE). Keep it as simple and streamlined as you can, automate wherever possible and adopt continuous inspection strategies where viable.
3. Delight customers. JBGE is a good start, but given the increasing competition we all face we sometimes find that we need to do more than just meet our customer's expectations, we need to exceed them. 
4. Optimize flow. DoR and DoD are quality plays, greater quality results in less waste, smoother flow, quicker flow, and greater predictability. 

Did I miss any questions?  If so, I'd love to hear them.