Essencewas created by the Software Engineering Method and Theory (SEMAT) community and approved by The Object Management Group as a standard in 2014. The basic of Essence is that “provides the common ground for defining software development practices” (see [R1-ES]) and also is intended to build and maintain an open library and marketplace of software engineering practices and education materials (see [R3-SMMS] ).
Essence is important, (see specification [R1-ES]), because:
“Provide a common base that is useful for software engineering endeavors of all sizes (small, medium, and large)”
“Enable method building by the composition of practices, so that methods can be quickly assembled by a project team to match their needs, experiences, and aspirations. Allowing the method to start small and grow as needed”
“Support method agility, so that practices and methods can be refined and modified during a project to reflect experiences, lessons learned, and changing needs”
“Support scalability including from one product to many, from one team to many, and from one method to many”
In this article I show how to describe DAD Essentials. Essentializing DAD is different from similar endeavors because Disciplined Agile (DA) is not a prescriptive method/framework like Scrum/SAFe. Instead, DA is a toolkit based on similar goals as Essence in that it is generative – both provide building blocks from which you can tailor and evolve a process that meets your needs.
DAD Essentials
The role of this Essentials is to provide an overview of the DAD guidance and over the capabilities that could be developed for each significant aspect (“Alphas”) of an Agile & Lean process. What is really happening on Agile/Lean adoption (and in any process improvement) is developing & exercising of new capabilities. What is specific to Disciplined Agile is that it provides guidance for developing the capabilities that team & organizations needs for their specific context.
DAD Essentials are presented here in cards format using the OMG/SEMAT Essence Language constructs as Alphas, Patterns, Resources, Activities, Work Products (see Glossary section below). Each “card” has a name, a description, and a list of capabilities for your team or organization to develop.
Glossary
The following terms are used in Essence.
Alpha
An essential element that is relevant to an assessment of the progress and health of the software engineering endeavor.
Patterns
A generic mechanism / complex concepts that are made up of several other elements
Resources
A source of information or content.
Activities
One or more approaches for carrying out some work to be performed and can recommend actions on alphas and/or work products in order to and relevance this work.
Work Products
An artifact of value and relevance for a software engineering endeavor.
Full delivery life-cycles
Full, beginning-to-end, delivery life-cycle it is an explicit representation of how a software release will progress in time. The pragmatism and effectiveness of Agile (<> Waterfall) are based on realistic progress milestones with the evolution of working software toward a consumable solution. A team could have a reference lifecycle, but in a different context, they may need to use also other types of life-cycles.
Capabilities to develop:
A reference life-cycle
Support for Iterative-Agile, Lean, Continuous Delivery life-cycles
Support for high incertitude deliveries
Support for long term roadmaps (product, business, technology)
Consumable Solution
Consumable solution is more than working software. Consumable means that we meet stakeholder needs in the context constraints and it is usable, desirable, and functional. A solution implies that, as needed, we:
Develop high-quality software
Provide new or upgraded hardware/platform
Change the business/operational processes which the stakeholders follow
Change the organizational structure in which our stakeholders work
Update supporting documentation
Capabilities to develop:
Understand Consumable Solution
Build using DAD guidance the Consumable Solution across life-cycle milestones, considering various life-cycle and practices options depending on context
Adapt to Context
DA supports two principles that motivate you to adapt your approach to your context:
Context counts. People, teams, and organizations are all unique – leads us to a critical idea that your process and organization structure must be tailored for the situation that you currently face.
Choice is good. Different contexts require different strategies – teams need to be able to own their own process and to experiment to discover what works in practice for them given the situation that they face. This is why the DA framework presents people with choices through the application of process goal diagrams.
Select practices per process goals in a given context and life-cycle
Use DAD Library of practices & context options guidance
Make experiments to see what works in your context
Core Agile Practices
Core Agile Practices will help you have a Lean process: they address the main sources of waste and have multiple benefits at the same time. It is not a coincidence that XP is based on some of them, Disciplined Agile and Agile Modeling refer them as critical practices. (See also references from Mary Poppendieck / Tom Poppendieck in “Lean Software Development”). Some core practices are:
XP practices: Small Releases, Pair Programming, Simple Design, Refactoring, TDD, Coding Standards, and more.
DA/Agile Modeling practices: Requirements/Architecture Envisioning, Architecture Handbook, Model Storming, Rolling Wave Planning, and many more.
Method free practices: Clean Code/Architecture.
Interestingly, Essence describes in detail several dozen core agile practices in detail whereas Disciplined Agile puts several hundred agile, lean, and even traditional core practices into context. This is one of several reasons why Essence and DA are complementary to one another.
Capabilities to develop:
Understand how and why you will eliminate waste
Context usage: Core Agile Practices are not “best practices” and we still need to know the trade-offs and options for different context (See Adapt to Context alpha). Make experiments to see what works in your context
Teal Teams and Organizations
Optimize the whole: the Organization (and its constituents teams) represents that “whole” where the work optimization really makes sense. In Reinventing Organizations, Frederick Laloux presents the historical evolution of organizations from tribal to modern agile approaches:
Green (Post-Modern/Information): Value-based, consensus/participative style
Teal (Self-Organizing/Adaptive): Cellular organism with evolutionary purpose
Disciplined Agile use this model and propose as a goal the Teal organization (or at least Green): cellular, self-organizing, adaptive, aware, with evolutionary purposes. Most likely, your organization is a “rainbow” (e.g. Orange/Green/Teal). Context counts, different teams faced different situations and you can choose your strategy. You want to be at least Green because that will provide – through a participative & collaborative style – a solid foundation for further process improvement. Also, the DA Principle, Be Awesome has some expectations:
Treat people with respect, honesty, be reliable and open
Willingly collaborate with others
Capabilities to develop:
Teams will offer psychological safety with clarity about roles and responsibilities
Cross-functional skills teams, where T-skilled “generalizing specialist” is the most pragmatic, effective, efficient approach.
Use collaborative work to envision, look ahead and just-in-time clarifications.
Collaborative and continuous process tailoring and improvement, not only on retrospectives
The purpose of the Continuous Improvement process blade is to enable people within your organization to easily share their improvement learnings with one another in a systematic way. The technique of Guided Continuous Improvement (GCI) shows teams how to leverage the DA toolkit to speed up their process improvement efforts.
Capabilities to develop:
Continuous improvement must be explicit and fundamental
A base support for improvement should be running: life-cycles, collaborative work, retrospectives
Kaizen strategy: continuous improvement should always be running in small steps and experiments. This lean strategy is fundamental for addressing problems complexity & incertitude.
apply the DAD toolkit to adapt to context (see corresponding alpha) for process goals as: selecting a life-cycle, forming the team, addressing changing stakeholders needs.
hire/listen experienced coaches
make progress on adopting Core Agile Practices
have explicit goal/guidance for Evolving your WoW
An Agile method such as Scrum, or a framework such as SAFe, can be a good start but it will have too few guidelines for choosing your way of working (WoW) in context or too little guidance for core Agile practices. DA provides the guidance for evolving your WoW and Essence the details regarding core practices. As Ivar Jacobson likes to say, this will enable you to break out of “method prison.”
Patterns
DA Principles
Delight Customers, Be Awesome, Pragmatism, Context Counts, Choice is Good, Optimize Flow, Enterprise Awareness
Agile & Lean Principles
DAD use Agile and Lean Principles intesively. Examples: Practices support and guidance, the Disciplined Agile Manifesto, Lean and Agile life-cycles etc.
Collaborative, Cross Functional Teams
Collaboration is a fundamental Agile and human value, and DAD supports that with several practices. Also, DAD promotes T-skilled “generalizing specialist” as an effective, pragmatic approach of cross-functionality.
Pragmatic Agile Roles
The DAD toolkit suggests a robust set of roles for agile solution delivery, roles that work well in real practice. DA propose also some secondary roles (less used, temporary or at scale)
Process Goals
DAD toolkit takes a light-weight, goal-driven approach to support adapt/tailor WoW to context. While process capabilities (goals) remain the same, you can use different choices and select different practices in a given context.
Scaling Factors
The context will permanently change and the Scaling Factors are significant aspects that will drive tailoring your element reflect the situation that you face.
Resources
Guidance, Adapt to Context: Select Life-cycles
Solution delivery teams face different situations so one lifecycle will not fit all. DAD offers more options for Agile and Lean life-cycles and appropriate guidance:
For each factor of process goals, DAD toolkit proposes more options of practices with guidance about efficiency and tradeoffs in context.
Library of Practices
DAD toolkit offers a library of practice including both Core Agile Practices and options for each process goals.
Agile Modeling
Agile Modeling Core Practices are art of the DAD toolkit and where developed as complement to XP.
Agile Data
Agile Data Core Practices are used by the DAD toolkit
Work Products
Consumable solution Increment
The basic element for measuring progress. Also referred as “working software” or “product increment”. See “Consumable Solution” alpha for differences.
Work Items Representations
Is not reduced to Product Backlog: we can use Work Item List (improved backlog concept), Work Item Pool or others.
Definition of Ready
Eliminate waste: streamline the flow evolution from incoming work to WIP. DAD practices: Look Ahead Modeling and Look Ahead Planning.
Definition of Done
Eliminate waste: advance without technical debt, avoiding re-work and unexpected problems and interruptions.
Activities
Selecting a life-cycle
Team activity: each release has a life-cycle choice. Preserving the one from the previous release or a model from an Agile Method (or even Waterfall) is also decision. Guidance & past experience will help.
Selecting practices
per Process Goals
Team activity: for each process goals the team will have its own choices. Preserving practices from previous releases or selecting others from some Agile methods is also a decision. Guidance & past experience will help.
Look Ahead collaboration
Looking ahead variants: envisioning the release, iteration look ahead and opportunistical look ahead before or inside the iteration. DAD offers collaborative practices for all of them and not only for iteration and daily planning.
Just in time collaboration
Just in time collaboration to clarify requirements, solution or other aspects. Example of practices: Pair Programming, Model Storming.
Lightweight Milestones Reviews
You can go beyond prescribed iteration level review in several ways. At the release level, you have the ones for the life-cycle milestones, including the one for proven architecture. At a smaller level, especially if you get the working software faster you could have automatic reviews (see Automatic validations).
Retrospectives
Improvement meetings, fundamental for continuous improvement. Collaborative work makes them effective.
Automatic validations
Include more kind of validations: automatic tests, automatic design check, and others. Fundamental for continuous integration, continuous delivery, and also for any form of often delivery/small releases.
Acknowledgements
I want to thank Scott Ambler who started this Essentializing DA initiative and collaborated with SEMAT from 2009. Scott helped me with feedback and review of current materials. DA Essentialization began with the example of the DB Refactoring technique earlier this year.
Use of Essence – Kernel and Language for Software Engineering Methods Specifications is the subject of Term and conditions & Notices found at https://www.omg.org/spec/Essence/1.2
Choosing your way of working (WoW) isn’t just a one-time event, instead it is an ongoing effort. Figure 1 shows the workflow for choosing and then evolving your WoW. In our previous blog, Choosing Your Initial Way of Working (WoW), we worked through the left-hand side of Figure 1. In this blog we explore how a team evolves their WoW via a series of experiments, hopefully ones that are guided by the Disciplined Agile (DA) toolkit.
Figure 1. The workflow for choosing and evolving your WoW.
As you can see in Figure 1, evolving your WoW is a two-step process at a high-level. First, you identify that you have a potential issue with your current WoW and second you experiment with one or more potential improvements that you believe will address the issue that you’ve identified. And of course you repeat this strategy whenever needed.
Identify Potential Process Issue
There are various ways that a team can identify a potential process issue:
Retrospectives. Retrospectives are a strategy for a team to reflect upon what is working well for them and what isn’t working so well. Some teams, particularly agile ones, will choose to hold a retrospective at the end of each iteration/sprint whereas lean teams tend to hold them on an as needed basis. Regardless, one of the outputs of such a working session is a list of one or more potential issues the team wants to address.
Someone recognizes there’s an issue. Sometimes it’s as simple as someone saying “Hey, I think that X is a problem. Is there anything we can do about it?”
Someone from outside the team points it out. It can also be as simple as someone from outside of the team – one of your stakeholders, a colleague on another team, a leader within your organization, or others – identifying a potential issue.
The point is that there are multiple ways that potential issues are identified. So what are you going to do about them?
Experiment with Potential Improvement(s)
For any process issues that you believe you can address, the next step is to experiment with one or more potential solutions. Experiment? What?!?!?!?! That’s right, experiment. Any given practice works well in some situations but not in others. Just because a technique worked well for another team, maybe even one that you’ve worked on in the past, that doesn’t mean that it will work well for your team in the context of the situation that you currently face. There is no such thing as a best practice, regardless of the endless marketing you may have heard telling you otherwise.
What you need to do as a team is to identify ways that you can potentially address an issue, narrow down your options, and see how well a given technique works for you by trying it out in practice. In other words, you need to run an experiment. Figure 2 depicts a continuous improvement loop, also known as a “kaizen loop,” where you choose a technique to experiment with, you try it for a sufficient amount of time to determine whether it works for you, and then you decide what aspects of the technique (if any) you should keep and which you should abandon. And if you’re enterprise aware your team will share your learnings with others. Guided continuous improvement takes this one step further by employing the DA toolkit to help identify potential new WoW for you to experiment with that is more likely to work for you, thereby increasing your team’s rate of improvement. Better decisions lead to better outcomes.
Figure 2. Guided continuous improvement.
An Example
Let’s consider an example. We’ve been working together as a team for several months, have released the initial version of our solution into production, and have been working on our next release for about a month. Our Team Lead has informed us that we’ve coming to the end of the funding for the team. When we formed the team we received funding for a 6-month project, following our company’s fixed cost approach to funding solution delivery teams. Our team expanded in size so that we could become a complete, whole team, and a side effect of that is that after a bit more than 4 months we’ve run out of money. This is a problem that the team needs to address.
Terry, our Team Lead, gathers the team to work through the issue. The first thing we do is discuss whether this is an issue that we can even influence. The Team Lead believes that we can because our organization’s leadership is very happy with our work and can see the value in the product that we’re working on. Because they have been receiving advice from an Executive Agile Coach they are beginning to realize that the way that they fund teams needs to evolve. Terry believes that our team is in a position to suggest, and then experiment with, a new approach to funding.
As a team we discuss what we need to do, realizing that there are really two issues commingled here: First, we’re funding a project, not the actual team. Second, we’re taking a fixed-price approach. Carlos, our Agile Coach, suggests that we review the options captured by the Secure Funding process goal, the goal diagram for which is shown in Figure 3. It indicates that both project-based funding and fixed-price funding are the least effective options for agile teams, and more importantly it also indicates that there are better options available to us. We look up the trade-offs associated with the options in our copies of Choose Your WoW! and after a bit of heated discussion agree that we should suggest to our management team that we adopt a stage-gate funding strategy for a product (long-lived) team. Several of us wanted to push for a time-and-materials (T&M) approach, but we felt that would be a future improvement that we could experiment with once we’re successful with stage-gate funding.
Figure 3. The Secure Funding process goal diagram.
Terry, with the support of Polly (our Product Owner), manages to convince our senior managers to experiment with a new approach to financing. Terry and Polly were able to describe the trade-offs associated with both the existing approach to funding and their suggested new approach. Interestingly, their suggestion was whole-heartedly supported by Florinda our finance officer. She’s been concerned for several years about the way that IT projects have been funded, and is eager to move from a cost-based funding model towards one focused on investing our company’s money wisely. Our team was given the go-ahead to try the new funding strategy.
Sure enough, we run the experiment with stage-gate funding of a product team and it works well. Our “stages” were three months in length, and after two rounds of such funding we successfully experimented with a T&M approach as we’d originally hoped.
