In this posting we explore the goal-driven aspect of the Disciplined Agile (DA) toolkit. This goal-driven approach enables DAD to avoid being prescriptive and thereby be more flexible and easier to scale than other agile methods. For example, where Scrum prescribes a value-driven Product Backlog approach to managing requirements DAD instead says that during construction you have the goal of addressing changing stakeholder needs. DAD also indicates that there are several process factors/issues surrounding that goal that you need to consider, and there are several techniques/practices that you should consider adopting to do so. DAD goes further and describes the advantages and disadvantages of each technique and in what situations it is best suited for. Yes, Scrum’s Product Backlog approach is one way to address changing stakeholder needs but it isn’t the only option nor is it the best option in most situations.
We start by describing how to visualize goals. We then summarize the goals called out by DAD, a topic we’ve written about in the past so we only cover this topic briefly here. We end with a summary of the advantages and disadvantages of a goal-driven approach over the more prescriptive approaches of older agile methods.
In the original DAD book we described process goals in a non-visual manner using tables which explored the advantages and disadvantages of the techniques associated with a process factor. Since we wrote that book both Mark and I have spent a lot of time helping people to understand what a goals-driven approach entails and we’ve found that many people respond well to visual representations of a process goal. Yes, the process decision tables are very important but a visual overview helps to provide context for the detailed information.
In the second half of 2012 we began developing a way to represent goals in a visual manner using what we call a goals diagram. A goals diagram, the notation for which is summarized in Figure 1, is in effect a form of decision tree. In Figure 1 you see that a process goal is indicated using a rounded rectangle and the decision points pertaining to a goal with normal rectangles. Process goals will have one or more decision points that you need to consider addressing, with most goals having four or five decision points although some have eight or nine. Each decision point is then addressed by two or more techniques/practices. Because there may be many techniques to choose from, we indicate “default” techniques in bolded italics. These defaults are good starting points for teams new to agile – they are almost always strategies from Scrum, XP, or Agile Modelling with a few Rational Unified Process (RUP) ideas thrown in to round things out. Some decision points you may choose not to address. Sometimes options are “ordered”, which is indicated by a upwards pointing arrow to the left of the list of techniques. What we mean by this is that the techniques appearing at the top of the list are more desirable from the point of view of agile and lean thinking and the less desirable techniques are at the bottom of the stack. Your team of course should strive to adopt the most effective techniques they are capable of performing given the context of the situation that they face. In Figure 1 the first decision point has an ordered set of options whereas the second one does not. Typically when the options are ordered you will only choose one of them whereas you MIGHT choose several options in unordered situations.
Figure 1. The notation of goal diagrams.
Let’s work through some examples. Figure 2 depicts the goal diagram for Explore Initial Scope, a goal that you should address at the beginning of a project during the Inception phase (remember, DAD promotes a full delivery lifecycle, not just a construction lifecycle). Where some agile methods will simply advise you to populate your product backlog with some initial user stories the goal diagram of Figure 2 makes it clear that you might want to be a bit more sophisticated in your approach. What level of detail should you capture, if any (a light specification approach of writing up some index cards and a few whiteboard sketches is just one option you should consider)? What view types should you consider (user stories are one approach to usage modeling, but shouldn’t you consider other views to explore the data or the UI)? Notice how we suggest that you likely want to default to capturing usage in some way, basic domain concepts (e.g. via a high-level conceptual diagram) in some way, and non-functional requirements in some way. There are different strategies you may want to consider for going about modeling. You should also start thinking about your approach to managing your work. In DAD we make it clear that agile teams do more than just implement new requirements, hence our recommendation to default to a work item stack over Scrum’s simplistic Product Backlog strategy. Finally Figure 2 makes it clear that when you’re exploring the initial scope of your effort that you should capture non-functional requirements – such as reliability, availability, and security requirements (among many) – in some manner.
Figure 2. Exploring the initial scope.
Figure 3 depicts one of the goals that you should address during the construction phase, in this case Address Changing Stakeholder Needs. This is an iteresting example for two reasons. First, it captures the key decisions surrounding the second of the 15 principles of the Disciplined Agile Manifesto, that of welcoming changing requirements. Second, it has a decision point that overlaps with that of another goal, in this case we indicate that your Work Item Management Strategy is important to consider for both this goal and Explore Initial Scope (see Figure 2).
Figure 3 makes the process factors surrounding how to address changing stakeholder needs very explicit. How are you going to prioritize changes? A business value approach is one option, the approach popularized by Scrum, but we’ve found that the risk-value approach promoted by Unified Process (UP) to be a more robust strategy that leads to greater chance of agile project success. There’s advantages and disadvantages to each technique so you’ll want to choose the one best for you. When are you going to accept the change? During the current iteration as Extreme Programming (XP) suggests or a future iteration as Scrum suggests? Do changes come directly from stakeholders or via a proxy such as a product owner or business analyst? How will your team elicit changes (via modeling, demos, …)?
Figure 3. Addressing changing stakeholder needs.
The advantage of visualizing goals as we’ve shown in Figures 2 and 3 is that it makes it very clear what process-related decisions you need to make and what options you have available to you. The disadvantage of this sort of diagram is that they get fairly big at times, as you can see. This effectively prevents us from taking the diagrams one step further to indicate the trade-offs associated with each technique and as a result you’ll still need the text tables we included in the DAD book for that.
The Goals of DAD
In the previous section we indicated that there are many goals called out by DAD, Figure 4 summarizes these goals, which have evolved slightly from what we published in the book (we refactored a few to make them more consumable). Notice how each of the three phases (Inception, Construction, and Transition) are described by specific goals. Also notice how some goals, such as Grow Team Members and Address Risk, are applicable throughout the entire lifecycle.
Figure 4. Goals throughout the lifecycle.
The Advantage of Goals Over Prescription
First and foremost, DAD is a process decision framework. One what that it achieves this through it’s goal-driven approach that guides people through the process-related decisions that they need to make to tailor and scale agile strategies to address the context of the situation that they face. Our experience is that there are several fundamental advantages to taking a goal driven approach to agile solution delivery. A goal-driven approach:
So far we’ve identified two disadvantages to DAD’s goal-driven approach when working with customer organizations. First, it makes the complexities of solution delivery explicit. Although some of us want to believe that the simplistic strategies of other agile methods will get the job done we inherently know that software development, or more accurately solution delivery, is in fact a complex endeavor in practice. Second, some people just want to be told what to do and actually prefer a prescriptive approach. DAD mitigates this problem a bit by suggesting default starting points (shown in italized bold text in the goal diagrams) but even this can be overwhelming for some people. Interestingly, when we were writing the book two of our 30+ reviewers were adamantly against giving people choices because they felt it was better to adopt a more prescriptive approach as we see in older agile methods.
We hope that this blog posting has given you some food for thought that you can leverage on your next agile project. Got Discipline?
Governance establishes chains of responsibility, authority and communication in support of the overall enterprise’s goals and strategy. It also establishes measurements, policies, standards and control mechanisms to enable people to carry out their roles and responsibilities effectively. You do this by balancing risk versus return on investment (ROI), setting in place effective processes and practices, defining the direction and goals for the department, and defining the roles that people play with and within the department.
Governance and management are two different things. Governance looks at a team from the outside, treating it as a system that needs to have the appropriate structure and processes in place to provide a stream of value. Management, on the other hand, occurs inside the team and ensures that the structure and processes are implemented effectively. The Disciplined Agile Delivery (DAD) process framework characterizes governance as an element of enterprise awareness from the team’s point of view because governance looks at the team from the outside.
It is easier to avoid your traditional governance and tell management that “agile is different” than it is to work with your governors to adapt your governance to properly guide the delivery of your agile teams. As we described in the book every organization has a necessary degree of governance and there are ways to make it especially effective on agile initiatives. It takes discipline to work with your governors to help them understand how disciplined agile teams operate and then discipline to accept and conform to the resulting governance process.
Our experience is that the most effective way to govern agile teams is to focus on collaborative strategies that strive to enable and motivate team members implicitly. For example, the traditional approach to motivating a team to provide good ROI would be to force them to develop and commit to an “accurate” project budget, and then periodically review their spending to ensure they’re on track. An agile approach would be to ask the team to provide a ranged estimate of what they believe the cost will be so as to set expectations about future funding requirements. Then the team works in priority order as defined by their stakeholders, visibly providing real business value through the incremental delivery of a potentially consumable solution. Costs are tracked via the team’s burn rate (the fully burdened cost of the people on the team plus any capital outlays for equipment or facilities) and value is tracked by the stakeholders’ continuing satisfaction (hopefully) with what the team is delivering for that cost. In short, a traditional approach often measures financial progress against a budget whereas an agile approach seeks to maximize stakeholder value for their investment by always working on the most valuable functionality at the time.
The DA toolkit includes several important agile governance strategies:
Many of the strategies described above are “standard” agile governance strategies, and a few are unique to DAD. It requires discipline to adopt and then execute on effective governance strategies, particularly in organizations where you already have a strong traditional governance program in place.
Because Disciplined Agile Delivery (DAD) addresses the full delivery lifecycle it explicitly addresses the effort required to transition your solution into production, or in the case of product teams into the marketplace. This transition effort may be referred to as release, deployment, or even the “end game”. For teams relatively new to agile this transition effort is a phase, or if you don’t like the term phase then an iteration which very likely has a different time frame than construction iterations. However, as we indicate in the DAD book, as teams gain more experience with agile and lean techniques the “transition phase” can be evolved into a “transition activity” with a little bit of discipline. That is the focus of this blog posting.
The DA toolkit is goal-driven, not prescriptive, and as a result it describes the transition effort in terms of goals:
It is straightforward to empirically observe that the complexity of your transition effort can vary depending on the context of your situation. For example, a simple standalone application such as a website can be easily deployed regularly because it effectively involves copying some files to a server (farm). Organizations in this sort of situation may choose to deploy their software many times a day. However, as we showed in the DAD book, for non-trivial enterprise deployments the transition effort can be significant. In fact, the November 2010 Agile State of the Art survey found that the average agile team spent six weeks in their transition efforts, with some respondents indicating spending one day or less and some indicating twelve weeks or more. There are several reasons why this happens:
Because of the potential complexities of releasing a solution in most mid-to-large sized organizations the deployment of solutions is carefully controlled. This is particularly true when the solutions share architectures and have project interdependencies, one of the reasons why DAD promotes the need for enterprise awareness within agile teams. For many reasons release cycles to your stakeholders are less frequent that you would like because of existing complexities within the environment. However, the ability to frequently deploy value to your stakeholders is a competitive advantage; therefore you should reduce the release cycle as much as possible. To do so requires a great degree of discipline in areas such as:
I’ve seen teams evolve transition phases of several weeks down to several hours through adoption of the disciplined strategies mentioned above. A disciplined agile team may start with relatively long Inception, Construction, and Transition phases and over time shrink all three down. Over time the Inception phase mostly disappears, particularly if you maintain team consistency between releases, and as I’ve argued in this posting the Transition phase shrinks to a very small activity. When deployment becomes inexpensive it enables you to have shorter construction phases and thus more regular releases – teams go from annual releases to bi-annual, then to quarterly releases, then monthly, weekly, and yes, sometimes even daily. Your team will need to choose a release cadence that makes sense for you.
These days it is fairly easy to observe multi-billion dollar companies, particularly e-commerce companies but even staid organizations such as financial institutions, deploy on a monthly, weekly, and even daily basis. If other organizations choose to work this way then why can’t you?
The Disciplined Agile Delivery (DAD) process framework includes an explicit Inception phase – sometimes called a project initiation phase, startup phase, or iteration/sprint zero – which is conducted before actually starting to build a solution. The goals of this phase include: clarifying the business problem that needs to be solved; identifying a viable technical solution; planning your approach; setting up your work environment and team; and gaining stakeholder concurrence that it makes sense to proceed with investing in the implementation of the chosen strategy. These goals are listed in the following diagram.
In the Disciplined Agile Delivery book we devoted a lot space to describing how to effectively initiate a DAD project. Unfortunately in our experience we have seen many organizations that are still new to agile treat this phase as an opportunity to do massive amounts of upfront documentation in the form of project plans, charters, and requirements specifications. Some people have referred to the practice of doing too much temporary documentation up front on an agile project as Water-Scrum-Fall. We cannot stress enough that this is NOT the intent of the Inception phase. While we provide many alternatives for documenting your vision in Inception, from very heavy to very light, you should take a minimalist approach to this phase and strive to reach the stakeholder consensus milestone as quickly as possible.
According to the 2013 Agile Project Initiation survey the average agile team invests about 4 weeks performing project initiation activities, including initial requirements envisioning, initial architecture envisioning, forming the team, initial release planning, and so on. Of course this is just an average, some respondents reported investing less than a week to do so and some reporting investing more than two months – the amount of time required varies depending on the complexity of the effort, your stakeholders’ understanding of their requirements, your team’s understanding of the solution architecture, whether this is a new solution or merely a new release of an existing solution, and many others.
If you are spending more than a few weeks on this phase, you may be regressing to a Water-Scrum-Fall approach. It takes discipline to be aware of this trap and to streamline your approach as much as possible. You can do this in several ways:
I think that it’s very clear that the secret to keeping Inception short is to have the discipline to know that you need to invest some time thinking your approach through but that you want to avoid getting bogged down in too many details. You need the discipline to do some planning but not too much. You need the discipline to do some modeling but not too much. You need the discipline to get going in the right direction knowing that the details will come out in time.
Techniques that shorten the time between doing something and getting feedback about it are generally lower risk and result in lower cost to address any changes than techniques with longer feedback cycles. Many of these techniques require agile team members to have new skills and to take a more disciplined approach to their work than they may have in less-than-agile situations. There are several common ways to shorten the feedback cycle that are common to agile software development that are adopted by the DAD process framework. These techniques, listed in order of immediacy, are:
This posting was modified from Chapter 21 of the forthcoming book Disciplined Agile Delivery to be published in June by IBM Press.