Project Management

Disciplined Agile Applied

by
This blog explores pragmatic agile and lean strategies for enterprise-class contexts.

About this Blog

RSS

Recent Posts

Hire for attitude, train for skills - But what skills?

Agile at 10: What We Believe

DoD and DoR: A Disciplined Viewpoint

Book review: Navigating DevOps Through Waterfalls

Your Organization is Likely a Software Company, Whether You Know it or Not

Hire for attitude, train for skills - But what skills?

Learn

 

Herb Kelleher's missive “hire for attitude, train for skills" is becoming more and more relevant as time passes. Kelleher, one of the founders of Southwest Airlines in the United States, believed in hiring good people and then supporting their learning journeys over time.  This is in stark contrast to the "gig economy" where you hire people with the skills to perform a specific job for a specified period to time so as to fill an immediate need.  Both are valid strategies in the right situation, but in this blog I'd like to explore Kelleher's philosophy.

Here are three important questions we need to answer about the "hire for attitude, train for skills" strategy if we're going to make it work in practice.  In particular, how do you identify:

 

Identifying Where Your Skills Are Deficient

This can be easier said than done.  Some organizations will have a robust People Management, often called Human Resources (HR), strategy in place that actively helps people to develop and then execute on a personalized learning journey. This is great if you have access to such a program, but even when you do there is still the challenge of identifying the skills and knowledge that you need to learn. Sometimes this is obvious, particularly if your organization already has strengths in the areas where you are currently weak, but not so obvious when a topic is relatively new to your organization or is rapidly evolving.

For example, let's assume that you're in a technical position on a solution delivery team and you need to expand your skillset around cybersecurity.  Unfortunately, you don't have easy access to someone with security expertise who would be able to guide you through how to learn more about security.  You could search online for some information, but what would you search for?  How can you tell which of the thousands of results you should focus on? A better option would be to start with the Disciplined Agile (DA) tool kit to see what it suggests regarding this topic so as to get you going in the right direction. 

Figure 1 shows two decision points, and their associated options, from the goal diagram for Disciplined Agile (DA) security process blade. Although there are many other decision points to consider, as you can see in Figure 2, let's focus for now on two that are directly pertinent to software developers: secure applications and secure data. You can see that there are several options that your team could adopt to develop secure applications, one of which, code review, you're familiar with but the rest you are not. At least now you have a short list of topics that you could explore in greater detail, narrowing your challenge down to something that is more manageable.  

Figure 1. Two decision points of the security process blade (click to enlarge).

Security techniques

 

Figure 2. The security process goal diagram (click to enlarge).

The Security process goal diagram

There is a much longer list of data security strategies provided, in fact this list is a bit daunting, because of the greater complexity of this aspect of cybersecurity. Quickly looking at the names it sounds as of many of the techniques are likely the responsibility of your data management team, although several sound as if they're more applicable for the solution delivery team that you're on.  In this case you decide to reach out to a friend on the data team to get their advice on where you should focus.

 

Identifying The Skills to Focus on Now

This decision is driven by the needs of your situation, the availability of options to gain the skill (training, coaching, and so on), and your preferences.  For example, if your team has an immediate need to improve their automated testing around security then learning about static code analysis techniques and tooling is likely what you need to focus on right now.  This may be quickly followed by learning about the other application security techniques in Figure 1.

 

Identify the Topics to Begin Exploring Now to Prepare for the Future

Many skills can't be learned quickly, or more accurately you need to understand the fundamentals of some topics before you can gain specific skills in them.  Once again, let's consider security.  It was a very big assumption on my part that you would have the background required to learn about static code analysis, penetration testing, or other application security strategies. What if you have very little knowledge about security or testing? Diving deeply into a specific skill likely isn't going to work out well in this case, your first step before doing so is to learn some fundamentals.  But how do you do that?

Luckily there is a lot more to the DA tool kit than just lists of techniques. When we describe process blades we work through four views:

  1. People. There are specialized roles, and responsibilities for those roles, associated with each process blade. For example, the asset management process blade describes the roles of asset manager and asset engineer whereas data management has roles such as data manager and database administrator.
  2. Mindset. The Disciplined Agile Mindset describes the principles, promises, and guidelines necessary for enterprise agility.  Each blade extends the DA Mindset with a collection of philosophies specific to that process area.  For example, Figure 3 overviews the mindset for disciplined agile vendor management.
  3. Flow. Disciplined Agile captures the flow of work in several ways, including life cycles, the DA FLEX value stream, and work process flows. For example, Figure 4 depicts a context diagram which overviews the high-level flow of disciplined agile research & development and Figure 5 shows the internal workflow of disciplined agile asset management.
  4. Practices. Practices are captured via goal diagrams such as the one for security in Figure 2 and described in tabular format, which you can read via the DA Browser.

 

Figure 3. The mindset of vendor management (click to enlarge).

Disciplined Agile Vendor Management Mindset

Figure 4. The external workflow of research & development (click to enlarge).

Disciplined Agile Research & Development workflow

Figure 5. The internal workflow of asset management (click to enlarge).

Disciplined Agile Asset Management workflow

 

We Need to Learn Continuously

Given the rapid pace of change within our current environment, many skills appear to have a short shelf life. The implication is that you want to know how to identify skills that are relevant now for the situation that you face and then pick those skills up quickly.  You've seen in this blog that the Disciplined Agile tool kit can help you with identifying the skills to learn.

Posted on: May 03, 2021 06:47 AM | Permalink | Comments (2)

Agile at 10: What We Believe

Categories: agile-manifesto, DDJ

Dr Dobb's Logo

NOTE: This article was originally published in the spring of 2011 in Dr. Dobb's Journal.  The magazine is unfortunately defunct now, and some of the material is now disappearing from the original site, including the original publication of this material. I was recently asked if I still had the original, so I have decided to repost this here.

What follows is my original, submitted article before copy editing.  So there may be differences compared with what was originally published.  For this publication I have removed some links which are no longer valid (hey, it's been 10 years).


What We Believe: A Non-Update to the Agile Manifesto

The Agile Manifesto had its beginnings in a meeting held in February 2001 at the Snowbird ski resort just outside of Salt Lake City.  To celebrate the ten year anniversary of this event Alistair Cockburn, who organized the original meeting, invited a group of agile luminaries back to Snowbird to discuss the future direction of agile.  The original 17 manifesto writers were invited, although not all were able to attend, and I was among the people who did participate.  In this newsletter I discuss what we concluded, how the event was organized, and some of the challenges faced by the agile community. 

 

What We Believe

First, let’s jump to what most people will perceive to be the end results. As a group, we agreed in the following four belief statements:

1. Demand technical excellence 

2. Promote individual change and lead organizational change 

3. Organize knowledge and improve education 

4. Maximize value creation across the entire process

Let’s dive down into what these belief statements imply.  The first statement, demand technical excellence, is a reflection both of the general values of the working group and also our frustration with what we’re seeing in practice – or more accurately not seeing in practice.  The Agile Manifesto exudes the philosophy that technical excellence is critical to your success in the software game, yet this requires both skill and discipline to pull off in practice.  Many agile teams do in fact manage this feat, as various DDJ surveys have shown over the years agile teams produce higher levels of quality on average.  However, the adoption rate of quality techniques such as test-driven development (TDD), refactoring across all architectural tiers, and following common development conventions aren’t as high as the agile rhetoric would lead us to believe.  There is room for improvement, and our hope is that this first believe statement will motivate agilists to do so. 

The second belief statement is to promote individual change and lead organizational change.  Once again, we’ve definitely come a long way as a community but we have further to go when it comes to improving our approach to software development. At the individual level I believe that we need to actively life-long learning, teach people to observe what works for them (and what doesn’t) in the situations that they find themselves in, and to then act on those observations. At the organizational level it is important to understand both the mechanics and the supporting human behavioral aspects of change processes and then invest the time and resources to effectively execute on them.  One aspect of this is to focus on the organizational ecosystem as a whole and not just on IT – in fact, many agilists are instead still mired in the details of software development and unable to see the enterprise forest for the development trees.

Organizing knowledge and improving education is the focus of the third belief statement.  Several people in the group voiced their frustration around the lack of a coherent and consistent agile body of knowledge.  In many ways this is a reflection of our times – the Internet makes it easy for people to publish articles, to post ideas in blogs, to tweet, to write wiki pages, and so on which in turn promotes a dispersed cacophony of voices.  This cacophony can be debilitating when you want resources which support teaching agile concepts, particularly at the college and university level but also at the individual learning level.  Heresy aside, perhaps an organization such as the Agile Alliance could start amalgamating some of this knowledge, and even go so far as to create an Agile Book of Knowledge (ABoK).  Minimally, it’s clear that the agile community could do more to reach out to universities and colleges to help them understand and teach agile more effectively.

Last, and certainly not least, is the fourth belief statement that we should strive to maximize value creation across the entire process.  One of the fundamental messages of the Disciplined Agile Delivery (DAD) process framework is that we need to look beyond the software construction focus of the Agile Manifesto to address the full solution delivery effort.  But this is clearly not the full picture either as there is much more to IT than solution delivery, and certainly much more to your organization that IT.  Some of the greatest challenges that we face with agile adoption are the business issues surrounding IT, particularly when it concerns financial issues such as funding delivery projects, IT governance, and understanding and interacting effectively with business stakeholders to name a few. As the lean community implores, we need to optimize the whole, and agile software development is only one small part of your overall organizational ecosystem. 

 

How it Happened

To help bring a bit of transparency to this event, I’d like to briefly describe how it was organized.  In early January 2011 Alistair Cockburn sent out invitations to a number of people whom he felt had added value within the agile movement over the years.  Not everyone was able to attend for various reasons, and perhaps a few people who weren’t invited should have been, but from what I could tell he brought together a group who was representative of the overall community.  I’d like to once again express my thanks to Alistair for taking the lead on this effort.

Luckily Alistair decided to hire professional facilitators, Robert Moir and Janet Danforth, to organize the event and I truly believe that we couldn’t have succeeded without the two of them.  A few weeks prior they called around to the confirmed attendees to discuss what they wanted to achieve when we got together.  We first saw the results of this the night before when we gathered in the evening for drinks and discovered that index cards containing questions had been distributed around the tables.   The goal of the cards was to stimulate conversation, not that we needed any help with that, and to communicate to everyone the range of issues to consider the following morning.

The working session itself was held on Saturday February 12th, with an idea generation session in the morning from 8:30 to 12:30 and a focusing session held in the evening from 5 to 7:30.    During the morning session we organized into seven subgroups, and we iteratively addressed seven topics – value; agile backlash/retrospective; technical/agile process; other communities and business/enterprise process; culture; education/training, and the future – which the index cards from the night before had been assigned to.  Each subgroup started with one of the topics and with sticky notes identified issues pertaining to that category.  Then the subgroups rotated to the next category to review the previous work and to add any new ideas that were missed previously.  We proceeded iteratively until each subgroup had contributed to each topic category.  

Over the break several of us chose to go skiing, the weather was perfect and the mountain offered a range of challenging ski runs – if you downhill ski then I highly suggest Snowbird.  In parallel many people decided to continue discussing the issues identified during the morning session.  The facilitators also took the opportunity to organize our work and to prepare for getting the group back together that night.

The goal of the evening session was to come to a consensus around a statement of our beliefs.  To do this we re-categorized the issues that we identified in the morning into things that we can make good headway on, things we don’t know how to address, and things that we believe can’t be addressed.  For example, better communicating the role of architecture in agile development is something we could make good headway on whereas the fact that many organizations rate their employees in part based on the degrees or certifications is something we’ll never be able to address as a community.  The issues which we believed could be addressed were clustered by topic, the topics were prioritized, and after much discussion and wordsmithing the four belief statements were identified.

 

The Elephants in the Room

Late in the morning we recognized that there were several issues that we were avoiding in our discussions.  This avoidance was due in part to the prevailing culture of the agile community, in part due to commercial interests of several people in the room, and in part because we were originally asked by Alistair to play nicely together.  So we invested some time to identify these “elephants in the room” so that they could be discussed.  I’ve listed these issues below, using the format of “elephantine issue – my thoughts on the subject” so that you can hopefully gain better perspective as to some of the issues holding back the agile community.  These issues are:

  • There are other effective ways of information discovery without writing code – Agile modeling, for example.  
  • Managers and management are bad – This is a common refrain of many programmers, including agile ones.
  • Politics within the community – There are many factions within the agile community, they don’t always agree nor do they even get along.
  • Failure to dampen negative behaviors – We often promote good behaviors within the agile community, such as allowing requirements to emerge over time, but do not push back sufficiently against poor behaviors such as documentation-based governance strategies.
  • Technical debt – Are we really paying down technical debt within organizations or merely talking about it?
  • Anarchism – Some of the alliances/organizations within the agile community promoting to be open efforts are little more than Ghaddafiesque groups in practice.
  • Hypocrisy – The reality of agile is often very different than the reality, something I’ve explored over the years via DDJ surveys.
  • Context and applicability – Many agile strategies are promoted as if they are the one best way of doing something, yet in practice they prove to work well in some situations yet very poorly in others.  The Situation Context Framework (SCF) is a contextual framework which may be used to address this problem.
  • Context gets in the way of dogma – Related to the previous point, it’s uncomfortable to admit that your one best strategy isn’t the only effective way of work, nor always the best.
  • Certification – As I pointed out in January, the agile community continues to build up integrity debt by tolerating questionable certification schemes.
  • Pretending agile is not a business – ‘Nuff said.
  • Abdicating responsibility for product success (to product owners) – Scrum’s product owner concept is useful in some contexts, but in practice it is a very difficult role to fill and reflects a serious lack of understanding as to the difficulty surrounding requirements management activities.
  • Agile Alliance – The AA showed great promise years ago and could potentially have lead the way in a true paradigm shift within IT, but in recent years has done little more than organize the annual Agile conference in the United States and provide funding for local agile events.
  • Role of architecture and design – Agile strategies for architecture and design modeling have been long overshadowed by programming practices and Scrum certification.  
  • Self organizing teams – The 2010 “How Agile Are You?” survey found that only 56% of “agile teams” were self organizing, and indication that many organizations are still struggling with this concept.
  • Elitism – The agile community needs to find ways to be more welcoming to project management professionals, data professionals, software architects, and many other specialist communities within IT.
  • Business value (what is it?) – We sure do talk a lot about providing business value, but don’t seem to have a good handle on what it actually means.
  • Scaling naivety – As I show in my work around the ASM, there’s a bit more to scaling agile than addressing the issues around large teams and geographically distributed teams, and it’s certainly more complicated than holding a “Scrum of Scrums” to coordinate everything.
  • Commercial interests – Perhaps we’d see more evolution in agile methods if we weren’t so focused on charging people to get certified in them.
  • Sensing failures – Although DDJ’s Project Success surveys over the years have found that agile project teams enjoy a higher success rate than traditional project teams it isn’t that much better and we don’t have a perfect track record.  
  • How do we know that we’re delivering value through software? – If we can’t define value surely we must also be struggling to measure it as well.

The four belief statements are not meant to be an addendum to the Agile Manifesto.  Several of us, myself included, felt that the manifesto should be updated to reflect what we’ve learned as a community these past ten years.  An equal number within the group, however, felt that the manifesto shouldn’t be updated.  It was clear that we wouldn’t reach any sort of consensus on this issue, let alone on what the changes should be, so we decided to leave well enough alone for now.  Interestingly, many of the “elephants in the room” could begin to be addressed by improvements to the manifesto.  Go figure.

 

>>HOT LINKS

I discuss The Agile Manifesto in detail in my article The Agile Manifesto Examined.

The Disciplined Agile Delivery (DAD) is a hybrid capturing strategies from Scrum, XP, Agile Modeling, lean software development, Kanban, and other agile/lean sources.

See Agility at Scale for more on the topic. 

Posted on: March 23, 2021 05:38 PM | Permalink | Comments (2)

DoD and DoR: A Disciplined Viewpoint

Great idea

As with many issues within the agile community, there is a bit of confusion around the pragmatic application of the Definition of Ready (DoR) and Definition of Done (DoD). In this blog I answer frequently asked questions (FAQs) around these two techniques.  These questions are:

 

What are the Definition of Ready (DoR) and Definition of Done (DoD)?

Figure 1 summarizes three key concepts that I will use throughout this blog:

  • Definition of Ready (DoR). The quality criteria which an input must meet or exceed to be considered acceptable by the team.  The team reserves the right to reject any inputs that do not meet the DoR.
  • Definition of Done (DoD). The quality criteria that a team promises that their outputs will meet or exceed. The basic idea is that something does not leave the team until it meets their DoD.
  • Disciplined Agile (DA) team. DA teams are semi-autonomous in that they find that they sometimes need to collaborate with others outside of the team to achieve their goals - the outputs of others are inputs into a DA team.  DA teams are self-organizing in that they decide how they are going to work together to fulfill their desired outcomes, producing outputs for their customers (which may be internal or external to the organization).

 

Figure 1. Defining DoR and DoD.

Definitions: Definiton of Ready and Definition of Done

 

Why do we want a Definition of Ready (DoR)?

As you can see in Figure 1, your team's DoR helps to protect you from poor quality inputs. When your inputs are insufficient they put your work at risk, forcing you to either rework the inputs to get them up to the level that you need or to cut the quality of your own work. By reducing rework, or in some cases hand-backs from the customer to the supplier, you help to optimize the flow between the teams.

For example, when you're making a salad, you very likely have a mental DoR for the quality of the ingredients.  You want fresh lettuce, fresh toppings, a nice dressing, and so on.  But what if the lettuce isn't sufficiently fresh?  Perhaps you cut off some of the spoiled leaves, forcing you to do extra work to get it to the point where it's usable.  Or perhaps the lettuce isn't quite as crisp as you would prefer, reducing the overall quality of what you produce and reducing your enjoyment of it.

 

Why do we want a Definition of Done (DoD)?

As you can see in Figure 1, your team's DoD helps to protect your customers from shoddy work. It is effectively an agreement between you and your downstream customers that puts a lower limit on the quality of your outputs. Your DoD helps to ensure the quality of your outputs, which in turn improves the predictability of your team in that your outputs are consistent and it helps to optimize the flow between teams due to reduction in hand-backs.

Once again, consider making a salad. If you produce a salad that is below your usual level of quality you will likely disappoint whomever you made it for.  If the salad is for you perhaps you can live with that.  If the salad is for your children perhaps you will have to endure their complaints, or even risk that they won't eat it at all.  If the salad is for a customer in a restaurant, you risk them sending it back forcing you to redo your work or they may choose to never frequent your restaurant again.

 

Can we define a standard DoR/DoD?

Sort of.

The challenge that we face is that our teams consume and produce a multitude of things, from tangible products to intangible concepts. And teams are unique, using those things in different ways.

For example, salads are the only item in my cooking repertoire.  I can make a wide variety of entrees, including spaghetti, grilled stakes, soup, and many more.  I can even make a range of deserts and drinks (I was a bartender at one point during university). Clearly the DoD for a salad, for a T-bone steak, for an apple pie, and for a margarita will vary.  There will also be similarities too, such make something delicious, make it look interesting, and don't poison people.

The implication is that any DoD or DoR standard will need to focus either on commonalities between the things that you produce or consume or be a checklist of potential issues to consider when formulating your DoR/DoD.

 

Must a team have a DoR or a DoD? Both?

In DA we believe that teams should choose their own way of working (WoW), including whether or not they want to adopt DoR and DoD as techniques. Because it's your choice, in the strictest sense of the question, the answer is no. DA isn't prescriptive that way, but as you learned above it's a very good idea to have a DoR and a DoD.

 

How do DoR and DoD relate to one another?

The outputs of one team are the inputs to other teams. These "teams" may be external to your organization.  For example, the lettuce that I used to make a salad came from a grocery store, which in turn came from a vegetable distributor, which came from a farm. The salad itself may be input into someone's overall meal.

Figure 3 summarizes the relationship between DoD and DoR, where the upstream team is the supplier to the downstream team, their customer. There are three situations to consider:

  1. The supplier DoD is insufficient (DoD < DoR). In this case the quality of the work being produced by the upstream team does not meet the needs of the downstream team.  Either they will need to negotiate respective changes to their DoD and DoRs to get them aligned, or the downstream team will need to find a new supplier that meets their needs. 
  2. The supplier DoD aligns with the customer DoR (DoD = DoR). The supplier perfectly meets the needs of the customer, in other words the output of the supplier meets the minimum quality criteria of their customer.
  3. The supplier DoD exceeds the customer DoR (DoD > DoR). On the one hand this is potentially wasteful because the suppler is doing more than they need to do, they are effectively injecting waste though over producing (gold-plating). However, on the other hand, this provides the opportunity to delight your customers because you've exceeded their expectations.  

 

Figure 2. Examining the relationship between DoD and DoR.

The relationship between DoD and DoR

What must a DoD minimally cover?

Sometimes your team produces outputs that are used by different teams in different ways, and each of those teams has their own DoR. The implication is that minimally your DoD is the union of your customer's DoRs for what you produce for them. What I mean by this is that your DoD must cover all of the criteria for all DoRs for downstream teams.

Consider the example in Figure 3. The output of the upstream team is being used by three different downstream teams. The DoR for Downstream Team #1 has two criteria, A and B. Similarly, Team #2 has criteria B and E and Team #3 A C and D. The implication is that the DoD for the Upstream Team must minimally address the combination of the three downstream teams, in this case A, B, C, D, and E.  

Figure 3. Your DoD is the union of the DoRs of your customers.

DoD is the union of your customer DoRs

Sometimes the minimum is only a start when it comes to quality. The minimum will hopefully satisfy your customers, but it might not delight them.  The challenge is that if another supplier comes along with a better offering, perhaps in this case A' that extends A to provide higher quality, then you risk losing your existing customers to that new supplier because they've gone above and beyond your offer.

Another strategy, described below, is to have multiple DoDs, one for each product version.

 

Do we need to negotiate our DoR and DoD?

It depends.

My gut reaction is to say yes, because that would be the smart thing to do. But you're not always in a situation where negotiation is possible.  Consider the following scenarios.

  1. The supplier is an external organization. You may be able to negotiate, particularly if you're a valued customer, or at least make suggestions.  You may also ask for a high-quality option from them. Or you may choose to find another supplier.  For example, if I'm at the market buying lettuce for my salad and I don't like the quality of what I see, I can ask someone there if they have anything fresher in the back, I could look at another type of lettuce or leafy green such as spinach, or I could go to another market.
  2. The customer is an external organization. This is the reverse of the previous situation.  Are you willing to act on the requests and feedback of your customers?  
  3. The two teams are in the same organization. Are we willing and able to negotiate with one another? Are we allowed to do so?  Does the supplier in this situation have the flexibility to act on suggestions and thereby evolve what they produce?  Does the customer in this situation have the flexibility to evolve what they'll accept?

 

What if we can't come to an agreement?

It depends.

This is an issue when:

  1. You're the customer of a single-source supplier. In the short term that's a problem because you don't have the option to go elsewhere.  In the long term you may be able to motivate another organization to get into this line of business, or perhaps even do so yourself. 
  2. You're the single-source supplier. Although you may feel safe because you've "locked in" your customers, this is a risky long-term strategy for you.  As soon as another option becomes available, particularly if it's seen as a better one, you risk losing customers. Organizations that don't focus on delighting their customers may soon find themselves in trouble.
  3. The supplier and customer are forced to work together. Sometimes you're told by senior leadership that "you two groups need to work together, figure it out." This can be tough when your history of working together is poor, when the fit between your teams isn't very good, or when there is little apparent value in doing so. In this case, negotiating your DoR and DoD respectively will be an important part of "figuring it out."

 

What happens when our DoR or DoD breaks?

Learn from the breakage.

Your DoR/DoD breaks when it is no longer sufficient.  Perhaps the upstream supplier changes what they produce without first negotiating with you.  Perhaps the downstream customer evolves their criteria for what they require. Regardless, you should treat this situation as a learning opportunity which very often leads to improvement.

 

What happens when a team produces more than one thing, or more than one version of a thing?

If they are different offerings, then you need a DoD for each one.

For example, we should have a DoD for the salad that we produce. If we also produced chicken stir fry, then there would be a DoD for that too.  There may be overlap in the DoDs, for example we may have criteria around not poisoning people with our cooking, but there would also be differences between them because salads and chicken stir fries are very different things.

Similarly, if you have more than one version of something than you need a DoD for each version. Earlier we talked about how a DoD for a given output needed to encompass all the DoRs of downstream customers.  Another option would be to refactor your output into different versions, one for each customers, so as to meet their exact needs.  This increases your overhead, even if you have a clever configuration strategy you still need to manage the configurations, but has the advantage of being able to offer customized offerings to customers.

 

Figure 4. Versioning your DoD to meet specific customer needs.

DoD with multiple outputs

 

Are DoR and DoD quality gates?

Yes.  

To be more accurate, they are a form or type of quality gate, the implementation if which is determined by the needs of the situation that you face. Context counts.

 

Aren't quality gates non-agile?

This question deserves a multi-part answer:

  • Why do you even care about this?  Your true goal should be to be as effective as possible, not to be agile.
  • It depends on whether your situation requires some form of quality gate.
  • It depends on how you implement the quality gates that you do need. 

Once again, consider the DoD for the salad:

  • If it's just for me, my "DoD inspection effort" will be non-existent because I will have made something that I like in the first place.  
  • If I'm making it for my daughter I've got a pretty good idea about what she'll eat.  At most I'll give the salad one last quick look before putting it in front of her.
  • If I'm a cook in a restaurant I've likely been coached on how salads are made at there so have a pretty good understanding on what to do, and worst case I would reach out to a colleague for advice. But that's not enough. The waiter picking up the food to bring to the customers will also look it over before bringing the salad out, and will pass it back to me if they feel I haven't done it well enough.
  • If I'm making it for a fast-food restaurant then I've likely been trained in exactly how they want salads made - one of their competitive advantages is consistency of product throughout their chain.  There's likely a detailed checklist for exactly what ingredients to use, in what order, and how much to use. There's the official salad packaging to use, and there's likely defined time constraints that I need to conform to.  All of this will be well documented. The manager is likely keeping an eye on what I'm doing, particularly when I'm new to that task, and there may even be some sort of artificial intelligence (AI) system that is visually monitoring the food preparation to ensure that the salad meets the DoD.

Context counts.  The implementation of your DoD, and similarly your DoR, varies by the situation that you face.

 

How should we document the DoR/DoD?

I always recommend an agile approach to documentation:

  • Don't write detailed prose when a list of concise points will do.
  • Don't list concise points when a philosophy or principle will do.
  • Don't set a philosophy/principle when automation will do. 

 

How should we implement the DoR/DoD check/inspection process?

The way that you implement the quality gate check/inspection for DoR/DoD depends on your situation.  You have several options available to you:

  • Formal review. The inspection may be a manual and potentially onerous process, involving one or more people explicitly inspecting our work. In the restaurant, this could be someone standing there with a clipboard to rigorously check the salad. Formal reviews certainly leave a lot of room for improvement, but may be the best that you can do at the present movement.
  • Informal review. In the restaurant, this was a waiter quickly looking over the salad before taking it out to the customer.
  • Occasional, "spot" inspections. Every so often the product is checked for quality.  For example, in the restaurant the head cook may come by to look at what I'm doing when they have a moment to ensure that my work is of sufficient quality.
  • Automation. Frictionless, fully automated checking, is becoming very common in the software and manufacturing realms. For our salad, the AI monitoring system that visually inspected the salad before sending it out to a customer.  This is effectively frictionless (except of course when my salad fails inspection).  
  • Continuous inspection through the creation process. The inspection may effectively not need to happen in simple situations, such as when I made the salad for myself, because the checking against the DoD is happening during the implementation effort itself.  

And of course I could have walked through a similar example for applying a DoR on the ingredients going into the salad.

 

Does DoR/DoD apply outside of software?

Yes. DoR and DoD are fundamental concepts that apply to a wide range of products, both tangible and intangible, and services. 

Also, have I not belabored the salad example enough?

 

Aren't there similar concepts in the software world?

Yes.  In fact, this blog was originally motivated by a conversation that I had with Al Shalloway where I observed the relationship between supplier DoD and customer DoR was similar to that of superclasses and subclasses via the Liskov Substitution Principle (LSP). According to LSP a subclass should be more specialized than a superclass, similarly the customer DoR should be equal to or a subset of the supplier DoD.

There is also the concept of design by contract where you define the interfaces of software components, the contracts so that others know how to interact with those components. The implication is that you don't need to know the implementation details of the component, you just need to know what it does and how to interface to it.

 

How does DoR/DoD relate to team working agreements?

Your DoR/DoD definitions are an important part of your external working agreement.

Remember that there are two aspects of your team working agreement - the internal working agreement between team members that defines the ground rules for how you will work together, and the external working agreement that defines how your team will collaborate with others. 

As per the previous question, this is effectively design by contract for teams.

 

Why is this so complicated?

Your implementation of DoR/DoD only needs to be as complex as your situation warrants, as you saw in the examples above. The overall concept may seem complicated right now because I'm purposefully exploring all of the issues surrounding these two techniques. But as I discussed above, the way that you implement DoR and DoD is completely up to you. My advice is to keep this as simple as you possibly can.

 

Why is there a raging debate about DoR and DoD?

There are several reasons why people are still debating these concepts after all these years:

  1. People like to argue.  Nuff' said.
  2. Literalist thinking. Many people, particularly in the IT world where agile comes from, tend to think in literalist, or "black and white," terms.  As a result of literalist thinking, these people often struggle to recognize what they are actually doing in practice because it doesn't fit nicely into their desired categories.  For example, you'll often hear people claim that because they haven't written down a DoR or a DoD that they don't have one.  Yet in my salad example, you clearly have a DoR concerning the quality of the ingredients that you're willing to use even though it isn't written down.  You also have an unwritten DoD in that you're unlikely to serve a poor quality salad to your family, or in the case of a restaurant to your customers. Luckily you can typically reason with literalists, walking them through a few examples to help them understand that there are other ways of thinking about these concepts.
  3. People confuse automated processes, or continuous inspection, with non-existence. Automation can effectively remove the friction of the inspection process to zero. Even though your team is no longer investing effort to check against the DoR/DoD criteria they are in fact still being validated against. Just because you've built the inspection effort into your overall development process, perhaps via automation or perhaps via adopting the habit of taking pride in your work, doesn't mean that a DoR or DoD doesn't exist - it just isn't explicit.
  4. Purist thinking. In some cases the problem is with purist thinking that is based on an unrealistic vision of agile, such as the idea that quality gates aren't agile.  As you read earlier, that's clearly nonsense. Purist thinking often stems from a combination of literalist thinking and a narrow range of experiences applying agile in practice. One of the Disciplined Agile principles is Be Pragmatic, so we prefer to understand the advantages and disadvantages of a technique and then provide advice for how to apply them effectively in practice when appropriate.  
  5. The "agile teams are autonomous" myth. Some agilists, particularly purists, don't like diagrams such as Figure 2 and Figure 3 that explicitly show how teams collaborate with one another. Collaboration between teams is an incredibly common occurrence. Yes, there is a desire to form teams that are cross-functional, have sufficient resources, and have the required skills to get the job done and that's a wonderful vision.  Now let's be be pragmatic.  Your team very likely needed to work with finance to get funding, it very likely was initiated via some sort of portfolio management effort, you're very likely being governed, you may need to get advice from your security experts to ensure your work meets your organizational standards, you may need to work with your data team to access existing data sources within your organization, you may need to interact with your support/help desk team when your customers run into issues, and so on.  At best your team will be semi-autonomous, having periods where you're autonomously but other periods where you're collaborating with others. DA teams are enterprise aware and explicitly choose to work with other groups in your organization so as to be more effective.

 

What does Disciplined Agile (DA) advise?

As always, look to the DA mindset for guidance:

  1. Context counts.  Have a definition of ready (DoR) and a definition of done (DoD) that reflect the situation that you face. The salad DoR/DoD varied significantly depending on whether I was making it for myself, my family, or customers of a restaurant.
  2. Be pragmatic. Your implementation of DoR/DoD should be just barely good enough (JBGE). Keep it as simple and streamlined as you can, automate wherever possible and adopt continuous inspection strategies where viable.
  3. Delight customers. JBGE is a good start, but given the increasing competition we all face we sometimes find that we need to do more than just meet our customer's expectations, we need to exceed them. 
  4. Optimize flow. DoR and DoD are quality plays, greater quality results in less waste, smoother flow, quicker flow, and greater predictability. 

Did I miss any questions?  If so, I'd love to hear them.  

Posted on: March 18, 2021 07:14 AM | Permalink | Comments (9)

Book review: Navigating DevOps Through Waterfalls

Categories: book

Navigating DevOps

I recently had the privilege of writing the foreword for "Navigating DevOps Through Waterfalls" by Brent A. Reed, Willy Schaub, Mathew Mathai, and Lauren Mix with illustrations by David Hughes-Coppins.  Regardless of where you are on your DevOps adoption journey, even if you are far down the path and realize that you’re actually on a continuous improvement journey, this book has something for you.  I learned a fair bit by reading this, and I’ve been doing this for years.

Here is the text of my foreword:

DevOps has become table stakes for modern organizations.  DevOps must be baked right into your organizational way of working (WoW) if you are to thrive in the “new normal” of constant change.  DevOps enables your teams to become semi-autonomous and self-organizing, more disciplined in their WoW, and in a better position to improve how they address the needs of their customers. This book provides proven strategies for how to make this happen.

The authors present their DevOps adoption advice as a story.  This is a tale of a fateful trip, a trip that started from a tropic isle, aboard a tiny ship.  The mate was a mighty sailing man, the skipper was brave and sure, and five passengers set sail that day for a three-hour tour.  A three-hour tour!  But the weather started getting rough, the tiny ship was tossed.  If not for the courage of the fearless crew, they would have been lost!  Oh wait, that’s Gilligan’s Island, not this story.  My bad. 

This book is a story that is based on the hard-won experiences of the four authors after helping various organizations for years in their DevOps adoptions. DevOps has effectively become table stakes for your organization’s IT processes, all the more so given how COVID-19 has upended societies around the globe.  In the private sector the marketplace has become far more competitive, requiring companies to get better at sensing what their customers need and responding quick to fill those needs before someone else does. In the public sector the needs of their citizens have increased while the tax base has shrunk, implying that they need to do more with less.  My advice is to take this book seriously – although it is presented as a work of fiction the lessons in it are very real and very important.

Now that I think about it, many DevOps adoptions efforts turn out to be a lot like Gilligan’s Island.  Many organizations mistakenly believe at first that they are embarking on a short effort to adopt DevOps, or worse yet to “install DevOps.”  They hope that their journey will be straightforward and over with quickly, perhaps three to six months to Gilligan’s three hours.  Then they quickly find themselves in trouble, stuck on a transformation trip that they don’t know how to navigate. 

Why do DevOps transformation efforts run into problems?  It’s often because they don’t invest time to identify a compelling vision and instead head out into the transformation sea.  A critical lesson that the authors convey in their tale is that you must define, and then communicate clearly and consistently, both the what and the why of your DevOps transformation. Your objectives and the results you how to achieve become a driving force for your organizational success, and a driver for what you measure. What gets measured will get improved, and you very likely have a lot of things that need to be improved to fulfill your goals.   

There are other similarities between ineffective DevOps transformations and Gilligan’s Island. Many episodes centered around some sort of unlikely strategy for the castaways to escape their predicament, but their plan would invariably fail due to mis-execution.  Sometimes the professor’s attempts to cobble together the existing material available to him – bamboo, coconuts, and something that had recently washed ashore – would fail because they simply weren’t up to the task.  Sometimes the castaways would have different personal goals and were working at odds to one another.  And sometimes one of the castaways, usually Gilligan, would make a mistake and ruin the plan.

Just like the castaways were thrown into a situation that they were ill-equipped to deal with, many DevOps transformations are similarly poorly thought through.  When funding for new tooling isn’t available, and teams are forced to make do with what they have, this is the equivalent of the professor trying to cobble together a radio from coconuts and bamboo.  When teams don’t know the what and the why of your strategy, they work at odds to one another and waste organizational resources.  When people haven’t received training and coaching in new DevOps techniques and technologies they are apt to make mistakes that will undermine your organization’s efforts to escape from their existing WoW.

But I digress.  The entire predicament of having to escape from Gilligan’s Island could have been completely avoided had they only known what they were doing to begin with.  First, they would have acted differently had they known they were going out into a storm.  And let me be clear about this, DevOps transformations always get stormy.  The advice described in this book shows you how to avoid common problems that other organizations have suffered from, and how to navigate through them.  

Second, if they had a better boat and a crew prepared for the situation that they faced then they very likely could have successfully weathered the storm. This book works through the equivalent of building a capable crew by describing the mindset and the skillset that they need to succeed.  This includes business representatives that are actively involved in the effort and a supportive engineering process (or at least the beginnings of one). It also shows how to build a better boat by picking the right project(s) and developing a supporting DevOps infrastructure.

Third, you need some outside help.  Just like the castaways on Gilligan’s Island faced problems that they weren’t capable of overcoming on their own, so will you.  The transformation journey described in this book is supported by experienced change agents who coach the organization along the way. Without such help you will quickly find your transformation efforts running aground on the shoals of organizational complexities you don’t know how to avoid on your own.

Regardless of where you are on your DevOps adoption journey, even if you are far down the path and realize that you’re actually on a continuous improvement journey, this book has something for you.  I learned a fair bit by reading this, and I’ve been doing this for years.

Posted on: January 12, 2021 10:34 AM | Permalink | Comments (4)

Your Organization is Likely a Software Company, Whether You Know it or Not

Software code

Software is at the heart of every modern organization. Before COVID-19 you likely heard the phrase "software is eating the world" and certainly COVID-19 has accelerated the digital transformation efforts of organizations that were still in the process of coming around to this philosophy.  But what does this really mean?  I believe that there are several important implications to consider:  

  1. The majority of organizations are really software organizations (whether they know it or not). For example, a bank is a software company that makes money providing financial services to you. A grocery chain is a software company that makes money selling food to you. An insurance company is a software organization that makes money selling insurance to you. And so on.  None of these organizations would be in business today without sophisticated software-based systems. This has a profound implication for your organization’s processes – if your enterprise is a software organization, then to be successful your organization must be effective in the creation and operation of software that provides real business value to you and your customers. If your products or service offerings are mostly intangible in nature then you're a software organization.

  1. Software is critical in non-software organizations too.  There are still organizations that deal with mostly physical things.  For example, consider a construction company that builds houses, office buildings, roads, and other tangible things. They use computer-aided design (CAD) software to capture their architectural plans, enterprise resource planning (ERP) systems to manage their organization, project management software to organize their projects, and many more.  Even if they are “merely” purchasing commercial packages to do this they still have important integration, both technical and people-oriented, to perform. 

  1. Other aspects of your organization are still important. Having said all this, no organization is completely focused on software.  There are still sales, finance, vendor management, legal, and many other important functions performed within your organization. These functions must fit together and can always be improved upon.  

  1. It’s really about “software plus”.  Although a properly functioning heart is certain critical for your wellbeing, there’s more to you than just your heart.  Similarly, although software is at the heart of your organization you need to address far more than that to be effective in today’s environment.  This is why many of the process blades of the Disciplined Agile (DA) tool kit, the hexes in Figure 1, clearly address non-IT issues.

Figure 1. The Disciplined Agile tool kit.

The Disciplined Agile Tool Kit

This is why the Disciplined Agile (DA) tool kit is more sophisticated than the agile software development frameworks you may be familiar with.  With DA we choose to address the actual challenge that you face, not just the IT part of the challenge. 

Posted on: January 04, 2021 12:58 PM | Permalink | Comments (7)
ADVERTISEMENTS

Necessity is the mother of taking chances.

- Mark Twain