Essencewas created by the Software Engineering Method and Theory (SEMAT) community and approved by The Object Management Group as a standard in 2014. The basic of Essence is that “provides the common ground for defining software development practices” (see [R1-ES]) and also is intended to build and maintain an open library and marketplace of software engineering practices and education materials (see [R3-SMMS] ).
Essence is important, (see specification [R1-ES]), because:
“Provide a common base that is useful for software engineering endeavors of all sizes (small, medium, and large)”
“Enable method building by the composition of practices, so that methods can be quickly assembled by a project team to match their needs, experiences, and aspirations. Allowing the method to start small and grow as needed”
“Support method agility, so that practices and methods can be refined and modified during a project to reflect experiences, lessons learned, and changing needs”
“Support scalability including from one product to many, from one team to many, and from one method to many”
In this article I show how to describe DAD Essentials. Essentializing DAD is different from similar endeavors because Disciplined Agile (DA) is not a prescriptive method/framework like Scrum/SAFe. Instead, DA is a toolkit based on similar goals as Essence in that it is generative – both provide building blocks from which you can tailor and evolve a process that meets your needs.
DAD Essentials
The role of this Essentials is to provide an overview of the DAD guidance and over the capabilities that could be developed for each significant aspect (“Alphas”) of an Agile & Lean process. What is really happening on Agile/Lean adoption (and in any process improvement) is developing & exercising of new capabilities. What is specific to Disciplined Agile is that it provides guidance for developing the capabilities that team & organizations needs for their specific context.
DAD Essentials are presented here in cards format using the OMG/SEMAT Essence Language constructs as Alphas, Patterns, Resources, Activities, Work Products (see Glossary section below). Each “card” has a name, a description, and a list of capabilities for your team or organization to develop.
Glossary
The following terms are used in Essence.
Alpha
An essential element that is relevant to an assessment of the progress and health of the software engineering endeavor.
Patterns
A generic mechanism / complex concepts that are made up of several other elements
Resources
A source of information or content.
Activities
One or more approaches for carrying out some work to be performed and can recommend actions on alphas and/or work products in order to and relevance this work.
Work Products
An artifact of value and relevance for a software engineering endeavor.
Full delivery life-cycles
Full, beginning-to-end, delivery life-cycle it is an explicit representation of how a software release will progress in time. The pragmatism and effectiveness of Agile (<> Waterfall) are based on realistic progress milestones with the evolution of working software toward a consumable solution. A team could have a reference lifecycle, but in a different context, they may need to use also other types of life-cycles.
Capabilities to develop:
A reference life-cycle
Support for Iterative-Agile, Lean, Continuous Delivery life-cycles
Support for high incertitude deliveries
Support for long term roadmaps (product, business, technology)
Consumable Solution
Consumable solution is more than working software. Consumable means that we meet stakeholder needs in the context constraints and it is usable, desirable, and functional. A solution implies that, as needed, we:
Develop high-quality software
Provide new or upgraded hardware/platform
Change the business/operational processes which the stakeholders follow
Change the organizational structure in which our stakeholders work
Update supporting documentation
Capabilities to develop:
Understand Consumable Solution
Build using DAD guidance the Consumable Solution across life-cycle milestones, considering various life-cycle and practices options depending on context
Adapt to Context
DA supports two principles that motivate you to adapt your approach to your context:
Context counts. People, teams, and organizations are all unique – leads us to a critical idea that your process and organization structure must be tailored for the situation that you currently face.
Choice is good. Different contexts require different strategies – teams need to be able to own their own process and to experiment to discover what works in practice for them given the situation that they face. This is why the DA framework presents people with choices through the application of process goal diagrams.
Select practices per process goals in a given context and life-cycle
Use DAD Library of practices & context options guidance
Make experiments to see what works in your context
Core Agile Practices
Core Agile Practices will help you have a Lean process: they address the main sources of waste and have multiple benefits at the same time. It is not a coincidence that XP is based on some of them, Disciplined Agile and Agile Modeling refer them as critical practices. (See also references from Mary Poppendieck / Tom Poppendieck in “Lean Software Development”). Some core practices are:
XP practices: Small Releases, Pair Programming, Simple Design, Refactoring, TDD, Coding Standards, and more.
DA/Agile Modeling practices: Requirements/Architecture Envisioning, Architecture Handbook, Model Storming, Rolling Wave Planning, and many more.
Method free practices: Clean Code/Architecture.
Interestingly, Essence describes in detail several dozen core agile practices in detail whereas Disciplined Agile puts several hundred agile, lean, and even traditional core practices into context. This is one of several reasons why Essence and DA are complementary to one another.
Capabilities to develop:
Understand how and why you will eliminate waste
Context usage: Core Agile Practices are not “best practices” and we still need to know the trade-offs and options for different context (See Adapt to Context alpha). Make experiments to see what works in your context
Teal Teams and Organizations
Optimize the whole: the Organization (and its constituents teams) represents that “whole” where the work optimization really makes sense. In Reinventing Organizations, Frederick Laloux presents the historical evolution of organizations from tribal to modern agile approaches:
Green (Post-Modern/Information): Value-based, consensus/participative style
Teal (Self-Organizing/Adaptive): Cellular organism with evolutionary purpose
Disciplined Agile use this model and propose as a goal the Teal organization (or at least Green): cellular, self-organizing, adaptive, aware, with evolutionary purposes. Most likely, your organization is a “rainbow” (e.g. Orange/Green/Teal). Context counts, different teams faced different situations and you can choose your strategy. You want to be at least Green because that will provide – through a participative & collaborative style – a solid foundation for further process improvement. Also, the DA Principle, Be Awesome has some expectations:
Treat people with respect, honesty, be reliable and open
Willingly collaborate with others
Capabilities to develop:
Teams will offer psychological safety with clarity about roles and responsibilities
Cross-functional skills teams, where T-skilled “generalizing specialist” is the most pragmatic, effective, efficient approach.
Use collaborative work to envision, look ahead and just-in-time clarifications.
Collaborative and continuous process tailoring and improvement, not only on retrospectives
The purpose of the Continuous Improvement process blade is to enable people within your organization to easily share their improvement learnings with one another in a systematic way. The technique of Guided Continuous Improvement (GCI) shows teams how to leverage the DA toolkit to speed up their process improvement efforts.
Capabilities to develop:
Continuous improvement must be explicit and fundamental
A base support for improvement should be running: life-cycles, collaborative work, retrospectives
Kaizen strategy: continuous improvement should always be running in small steps and experiments. This lean strategy is fundamental for addressing problems complexity & incertitude.
apply the DAD toolkit to adapt to context (see corresponding alpha) for process goals as: selecting a life-cycle, forming the team, addressing changing stakeholders needs.
hire/listen experienced coaches
make progress on adopting Core Agile Practices
have explicit goal/guidance for Evolving your WoW
An Agile method such as Scrum, or a framework such as SAFe, can be a good start but it will have too few guidelines for choosing your way of working (WoW) in context or too little guidance for core Agile practices. DA provides the guidance for evolving your WoW and Essence the details regarding core practices. As Ivar Jacobson likes to say, this will enable you to break out of “method prison.”
Patterns
DA Principles
Delight Customers, Be Awesome, Pragmatism, Context Counts, Choice is Good, Optimize Flow, Enterprise Awareness
Agile & Lean Principles
DAD use Agile and Lean Principles intesively. Examples: Practices support and guidance, the Disciplined Agile Manifesto, Lean and Agile life-cycles etc.
Collaborative, Cross Functional Teams
Collaboration is a fundamental Agile and human value, and DAD supports that with several practices. Also, DAD promotes T-skilled “generalizing specialist” as an effective, pragmatic approach of cross-functionality.
Pragmatic Agile Roles
The DAD toolkit suggests a robust set of roles for agile solution delivery, roles that work well in real practice. DA propose also some secondary roles (less used, temporary or at scale)
Process Goals
DAD toolkit takes a light-weight, goal-driven approach to support adapt/tailor WoW to context. While process capabilities (goals) remain the same, you can use different choices and select different practices in a given context.
Scaling Factors
The context will permanently change and the Scaling Factors are significant aspects that will drive tailoring your element reflect the situation that you face.
Resources
Guidance, Adapt to Context: Select Life-cycles
Solution delivery teams face different situations so one lifecycle will not fit all. DAD offers more options for Agile and Lean life-cycles and appropriate guidance:
For each factor of process goals, DAD toolkit proposes more options of practices with guidance about efficiency and tradeoffs in context.
Library of Practices
DAD toolkit offers a library of practice including both Core Agile Practices and options for each process goals.
Agile Modeling
Agile Modeling Core Practices are art of the DAD toolkit and where developed as complement to XP.
Agile Data
Agile Data Core Practices are used by the DAD toolkit
Work Products
Consumable solution Increment
The basic element for measuring progress. Also referred as “working software” or “product increment”. See “Consumable Solution” alpha for differences.
Work Items Representations
Is not reduced to Product Backlog: we can use Work Item List (improved backlog concept), Work Item Pool or others.
Definition of Ready
Eliminate waste: streamline the flow evolution from incoming work to WIP. DAD practices: Look Ahead Modeling and Look Ahead Planning.
Definition of Done
Eliminate waste: advance without technical debt, avoiding re-work and unexpected problems and interruptions.
Activities
Selecting a life-cycle
Team activity: each release has a life-cycle choice. Preserving the one from the previous release or a model from an Agile Method (or even Waterfall) is also decision. Guidance & past experience will help.
Selecting practices
per Process Goals
Team activity: for each process goals the team will have its own choices. Preserving practices from previous releases or selecting others from some Agile methods is also a decision. Guidance & past experience will help.
Look Ahead collaboration
Looking ahead variants: envisioning the release, iteration look ahead and opportunistical look ahead before or inside the iteration. DAD offers collaborative practices for all of them and not only for iteration and daily planning.
Just in time collaboration
Just in time collaboration to clarify requirements, solution or other aspects. Example of practices: Pair Programming, Model Storming.
Lightweight Milestones Reviews
You can go beyond prescribed iteration level review in several ways. At the release level, you have the ones for the life-cycle milestones, including the one for proven architecture. At a smaller level, especially if you get the working software faster you could have automatic reviews (see Automatic validations).
Retrospectives
Improvement meetings, fundamental for continuous improvement. Collaborative work makes them effective.
Automatic validations
Include more kind of validations: automatic tests, automatic design check, and others. Fundamental for continuous integration, continuous delivery, and also for any form of often delivery/small releases.
Acknowledgements
I want to thank Scott Ambler who started this Essentializing DA initiative and collaborated with SEMAT from 2009. Scott helped me with feedback and review of current materials. DA Essentialization began with the example of the DB Refactoring technique earlier this year.
Use of Essence – Kernel and Language for Software Engineering Methods Specifications is the subject of Term and conditions & Notices found at https://www.omg.org/spec/Essence/1.2
The current state of Agile, with both its advantages and drawbacks, is significantly influenced by Scrum, the most widely used Agile method.
As a drawback, after some (easy to adopt) first steps in Agile introduced by Scrum, many teams have no clear vision of how to advance with the process improvements and with Agile. As a consequence, we could say that the current state of Agile is a stalemate.
A significant part of the parts missing in many Scrum-based Agile adoptions, such as XP (Extreme Programming) engineering practices, was described by Robert C. Martin in his article The Land that Scrum Forgot.
The Advantage of Scrum: First Steps in Agile
Beyond the serious existing criticism, Scrum offers at least the following advantages:
Agile has become a widely known approach because of a significant number of Scrum adoptions (or adoption attempts)
Scrum is easy to adopt
Scrum adoption offers some important quick benefits
This strong ability of Scrum on making the first Agile steps is too often underrated. The main advantage of Scrum practices is not related so much to its direct benefits, but more to the elimination of the side effects of many bad legacy practices that are thus being replaced. For example, if a team uses the Agile-enabled self-organizing power to make poor process choices and disregards the practices for collaborative work, it will have worse results that a Scrum-based team that offers a base for collaborative work.
Going Further
A team using Scrum could go beyond the stalemate situation by using the XP engineering practices–refactoring, TDD (Test Driven Development), Pair Programming, and other–and/or it can go even further with Disciplined Agile (DA). Disciplined Agile has more practices, makes the Agile habits explicit (as Agile life-cycles), and offers guidance support instead of prescriptions.
The following sequence for adopting Agile, could be common, but is not an optimum approach–to a certain degree it inherently results in a waste of resources and in delays:
First adopt Scrum for easy first steps
Later on adopt complementary XP practices
Finally adopt DA to fill the gaps and adapt the process to the context
From a DA adoption point of view, we need to find an optimized approach that must have the following two advantages:
It enables quick first steps (similar with Scrum)
It no longer represents a stalemate situation
The First Steps: a Base that Enables all the Others
DA offers a good learning-oriented process decision framework that will help the teams to make their own process decisions and further improves their process. However, the first steps would be the most difficult ones and the main questions that arise are the following:
“How should I start? What are the first factors that should be subjects of my choices?”
We are proposing the following Agile starting points:
Use life-cycle and iterations as a container for the process
Start working in a collaborative manner: non-solo work practices
Start persistent improvement by adopting retrospectives
In the following section we will explain how the use of these quick first steps as a base will enable all the other improvements.
Start Working in a Collaborative Manner: non-solo Work Practices
Agile-based process improvements and Agile retrospectives work only if the teamwork is based on continuous and active collaboration.
The simple and rather vague Scrum practices for team collaboration have the great advantage of consistently replacing the old habits of non-collaborative work. We can easily go further and get better results by adopting Agile Modeling/Disciplined Agile non-solo work practices, that mean working with others for (few examples):
The essential element is to offer guidance to the team in order start to practicing non-solo work as soon as possible.
Start the Improvement by Adopting Retrospectives
The improvement based on Agile and other sources must become continuous and permanent. The very first step is to start using retrospectives (per iterations, releases, and so on).
Containers for the Process: Life-Cycle and Iterations
The teams will need these containers as a “living space” for their process options and choices. Scrum uses the Sprint (aka Iteration) as the main work container. XP goes further and has an implicit release life-cycle.
In Disciplined Agile we can find a more robust approach, meaning that we have both the iteration and also more explicit agile and lean life-cycles, whereby its selection is the team’s choice.
Ok, but “How should I start”?
For an “agile first steps” team, here is an option for the main process containers:
The iteration – This is a generic container for not-very-advanced cases
As soon as possible and using proper guidance, the team should be aware of the process goals (DA makes these goals explicit) and how these goals are distributed along the life-cycle and iteration.
Using this initial framework – process goals across life-cycle and iterations– the team should start working collaboratively (non-solo work practices) and start using retrospectives and other instruments to decide how to improve and adapt their process. Make sure that this approach is permanent and the process will have further optimizations.
