Disciplined Agile

When a disciplined agile project or product team starts, one of the process goals which they will likely need to address is Identify Architecture Strategy. This is sometimes referred to as initial architecture envisioning or simply initial architecture modeling. This is an important process goal for several reasons. First, the team should think through, at least at a high level, their architecture so as to identify a viable strategy for moving forward into construction.  A little bit of up-front thinking can increase your effectiveness as a team by getting you going in a good direction early in the life cycle.  Second, the team should strive to identify the existing organizational assets, such as web services, frameworks, or legacy data sources, that they can potentially leverage while producing the new solution desired by their stakeholders.  By doing this you increase the chance of reuse, thereby avoiding adding technical debt into your organizational ecosystem, and more importantly you reduce the time and cost of delivering a new solution as the result of reuse.  You will do this by working with your organization’s enterprise architects, if you have any.  This is an aspect of Disciplined Agile’s mindset of working in an enterprise aware manner.

This is an important goal for several reasons:

1. You want to get going in the right direction.  Your team needs to have at least a high level understanding of how they’re going to build (or configure) the solution, they just don’t start coding.  This helps the team to avoid potential rework later in the life cycle.  It is critical in situations facing lots of technical complexity.
2. You need to secure funding.  In the vast majority of organizations, teams are asked fundamental questions such as what are you trying to achieve, how long will it take, and how much will it cost.  Having an understanding of your technical strategy is important input into answering those sorts of questions.
3. You want to leverage organizational assets.  Because disciplined agile teams are enterprise aware they work closely with enterprise professionals such as enterprise architects so that they can take advantage of any existing infrastructure and assets.  They also want to ensure that their solution reflects the overall goals of their organization (often captured in technical or business roadmaps).

The process goal diagram for Identify Architecture Strategy is shown below. The rounded rectangle indicates the goal, the squared rectangles indicate issues or process factors that you may need to consider, and the lists in the right hand column represent potential strategies or practices that you may choose to adopt to address those issues. The lists with an arrow to the left are ordered, indicating that in general the options at the top of the list are more preferable from an agile point of view than the options towards the bottom. The highlighted options (bolded and italicized) indicate default starting points for teams looking for a good place to start but who don’t want to invest a lot of time in process tailoring right now. Each of these practices/strategies has advantages and disadvantages, and none are perfect in all situations, which is why it is important to understand the options you have available to you.

Let’s consider each process factor:

• Choose the level of detail.  How much effort should you put into capturing your architectural strategy, if any at all.  A small co-located team may find that a high-level overview captured using whiteboard sketches is sufficient.  A team that is geographically distributed across several locations will find that it needs to at least take digital snapshots of the sketches and share them (perhaps via a wiki) or even capture the diagrams using a drawing tool such as Visio or OmniGraffle.  They may also find that they need to define the details of the interfaces to major components and services, a strategy often referred to as design by contract.  A team in a life-critical regulatory environment may choose to capture more details, even to the point of doing big design up front (BDUF).
• Explore technology architecture.  Technical aspects of your architecture may be captured via free form layered architecture diagrams, network diagrams, or UML deployment diagrams amongst others.
• Explore business architecture.  Business or domain aspects may be explored via conceptual domain models, high-level process models, and UML component diagrams amongst others.
• Explore user interface (UI) architecture.  For end users, the UI is the system.  The implication is that you had better architect it well, and more important ensure that it is usable/consumable.  You may explore the user interface architecture via a UI flow diagram/wireframe model, low-fidelity UI prototypes, or high-fidelity UI prototypes.
• Consider the future.  A critical thing is for your architecture to be able to accommodate change in the future.  The lean strategy is to consider these changes, and make architectural choices that best enable you to accommodate the changes when and if you need to, but DO NOT overbuild your solution now.  In short, think but wait to act.  To do this you need to consider the potential changes that could occur, often via “what if” discussions.  If you need to capture these potential changes you might decide to write change cases.
• Apply a modeling strategy(ies).  How will your team go about formulating their architecture?  Will they hold informal modeling sessions in an agile modeling space or formal modeling sessions Joint Application Design (JAD) strategy?  Or both?
• Select an architecture strategy. Will they identify a single technical strategy or several options which they will hope to explore and eventually choose from (or combine ideas from)?
• Identify a delivery strategy.  Will the team be building a solution from scratch?  Will they be evolving an existing solution?  Will they be evolving a purchased, commercial off the shelf (COTS) package?  Combinations thereof?

We want to share two important observations about this goal.  First, this goal, along with Explore Scope, Coordinate Activities, and Accelerate Value Delivery seem to take the brunt of your process tailoring efforts when working at scale.  It really does seem to be one of those Pareto situations where 20% addresses 80% of the work, more on this in a future blog posting.  As you saw in the discussion of the process issues, the process tailoring decisions that you make regarding this goal will vary greatly based on the various scaling factors.  Second, as with all process goal diagrams, the one above doesn’t provide an exhaustive list of options although it does provide a pretty good start.

We’re firm believers that a team should choose their own way of working (WoW), including their team structure, their work environment, and their process, to reflect the situation that they find themselves in.  When it comes to process tailoring, process goal diagrams not only help teams to identify the issues they need to consider they also summarize potential options available to them.  Agile teams with a minimal bit of process guidance such as this are in a much better situation to tailor their approach than teams that are trying to figure it out on their own.  The DA tool kit provides this guidance, particularly via Guided Continuous Improvement (GCI).

Recently on a LinkedIn discussion forum people were sharing their opinions about agile teams. Throughout that conversation I heard several statements which just didn’t ring true with me. In some cases I suspect that the problem was they were thinking they were agile when they clearly weren’t and in other cases I suspect people were sharing ill-founded rumours about how agile works. I heard that everyone on an agile team must be highly skilled, that everyone on an agile team is a “jack of all trades”, and that people on agile teams are working so fast that they have no time to learn. Let’s examine each myth/misunderstanding one at a time.

Myth #1: Everyone on an agile team must be highly skilled
Agile lore recommends that you build teams where you ideally have all the skills you need to get the job done, this is called being a “whole team”. This doesn’t always happen at first. Recently I worked with a team that was clearly short on testing skills but they were actively addressing that problem by bringing people into the team with those skills. They will also be pairing to spread the skills out once these new people join the team. Call me crazy, but doesn’t building a team with people that have the requisite skills a good strategy regardless of paradigm?

Myth #2: Everyone on an agile team is a “jack of all trades”
Disciplined Agile Delivery (DAD) suggests that people strive to be generalizing specialists (not generalists, not jack of all trades). This means that you have:

• One or more specialties (you need to be able to do something useful)
• At least a general knowledge of the software process and the business domain that you’re working in (this enables you to communicate more effectively with others and streamline collaboration)
• The desire to increase your knowledge and skills

Granted, this is different than how some people staff traditional projects where they have specialists and then incur an incredible amount of overhead to get the job done. When you are new to agile you very likely have many people who are specialists on staff, perhaps they are focused on being a business analyst, on being a tester, on being a programmer, an architect, and so on. That’s OK, as they say you go to war with the army that you have. Over time you will want to actively support, and motivate people, to learn new skills from other people on the team (see next point). It’s also important to recognize that although someone may currently be focused on a certain traditional role today, that in the past they held other roles and as a result may have a broader range of skills, albeit rusty in some cases, than they may give themselves credit for. Regardless of your situation people can easily gain new skills if they choose to.

Myth #3: People on agile teams are working so fast that they have no time to learn
Yes, agile teams focus on producing real value on a regular basis, and that is often perceived as working fast by people used to the less disciplined strategies of the traditional world. This is overwhelming at first for teams new to agile and it can seem that there isn’t time to learn or to even catch your breath. As you get better at working in an agile manner, as you “figure it out”, your team will settle on a rhythm that works for them.

Agile promotes iterative, collaborative, and experimental strategies. This promotes learning within the team, it doesn’t reduce it. Agile, and lean strategies in particular, expect the team to learn as you go. They’ll learn more about the domain, about the technologies they’re working with, about how to work effectively, and about each other. When people work together collaboratively, not alone at their own desks, they start to pick up skills from one another naturally. This is particularly true when the team adopts non-solo strategies such as pairing (I alluded to the value of pairing programmers and testers earlier). Of course coaching and mentoring are important to support learning as well as training.

I often run into people who are concerned about changing requirements, or evolving requirements if you like that term better, on software development projects (or product teams as the case may be). This is typically a reflection of their training and background in traditional software development strategies where changing requirements are usually perceived as a problem.

In agile we consider changing requirements to be a good thing and we even embrace the idea that this will happen.  In fact, one philosophy of Disciplined Agile Delivery (DAD) is that changing requirements are a sign of a healthy relationship with your stakeholders. Changing requirements indicate that:

• Your stakeholders are interested in what you’re working on
• Your stakeholders are thinking about what you’re producing
• There are feedback mechanisms in place that enable your stakeholders to change their requirements
• It is easy (hopefully) for stakeholders to change their requirements as their understanding of their needs evolve

So how do you support changing requirements in practice?  The Disciplined Agile (DA) toolkit includes the “Address Changing Stakeholder Needs” goal which guides you through understanding and selecting the right strategy for prioritizing requirements changes as they occur throughout the lifecycle.  These strategies include, but aren’t limited to Scrum’s Product Backlog Strategy, a more disciplined Work Item List strategy, or a lean Options Pool strategy.  In regulatory situations you may even want to consider a formal change management strategy.  DAD’s process goal-driven approach enables disciplined agile teams to tailor their strategy to meet the situation that they find themselves in, avoiding the challenges seen with methods such as Scrum that prescribe a single strategy (in this case Product Backlogs) that don’t work in all situations.

And of course, to minimize the pain of changing requirements you will need to adopt disciplined development practices such as:

• Writing high-quality code;
• Refactoring your work to keep it of high quality;
• Continuous integration;
• and even acceptance test-driven development (ATTD)/behavior driven development (BDD).

Recently on the DAD LinkedIn discussion forum we’ve had an interesting conversation going on that strayed into the effectiveness of various communication techniques.  This conversation got me to thinking a bit about an article that I first wrote over a decade ago, Communication on Agile Software Projects, based on material in my Agile Modeling book.  This article extends some of Alistair Cockburn’s thinking on the topic, summarized in Figure 1, which in turn extends a body of work called Media Richness Theory.  The basic observation is that the richer the communication channel – such as email, videoconferencing, or face-to-face conversation – the more effective it is in practice.

Figure 1. Comparing the effectiveness of communication techniques.

In the DAD book Mark and I said that the implication of Figure 1 was that given the situation you find yourself in that you should choose to use the most effective communication strategy available to you.  We thought that this was all that needed to be said, but since then we’ve come to realize that a better diagram is needed.  We developed Figure 2 which we believe communicates this strategy a bit more clearly.

Figure 2. Choosing the best communication technique for your situation.

Figure 2 indicates that that the viability of communication options available to you varies depending on the situation that you find yourself in.  For example, people on a co-located team can communication in pretty much any manner they choose.  Ideally they would choose face-to-face communication around a whiteboard or paper but there’s nothing stopping someone from writing a detailed document and then emailing it to someone sitting beside them.  When a team is geographically distributed they have fewer communication strategies available to them, unless of course they invest in travel so that they can be face-to-face.

Figure 2 also indicates strategies for capturing, or persisting, a conversation.  For example, a few people could have a conversation around a whiteboard, sketching the design of a screen perhaps.  When they’re done they might decide to take a digital snapshot of it so that they can retain the diagram to work on it later (many organizations still don’t provide permanent whiteboard space to their development teams).  They may even choose to have someone write up notes, perhaps on a wiki or in a word processor, and embed the snapshot(s) into those notes.

To summarize, when it comes to how people communicate with one another they have choices.  Disciplined agilists prefer to choose the most appropriate communication strategy for the situation that they find themselves in.

We’ve recently been asked how risk management is addressed on agile teams. This is an easy question to answer because risk management strategies are built right into the Disciplined Agile (DA) toolkit.

Let’s start with two definitions. First, a risk is an exposure to a potentially negative outcome. Risks come about from uncertainty. Second, risk management is the identification, exploration, and mitigation of risks. Risk management should be part of both your team management efforts, even on self-organizing agile teams, as well as part of your organizations governance efforts.

Regardless of your delivery paradigm, there are several common categories of risks faced by IT delivery teams:

• Business risk. This includes significant shifts in requirements due changing business priorities, often due to realization new market opportunities, in reaction to moves made by competitors, or because you are opening completely new market opportunities. Another common risk is an overly optimistic/aggressive schedule which motivates your team to take unfortunate shortcuts. A third type of business risk is a misaligned budget: either there isn’t sufficient funding to do the job effectively (e.g. there’s no travel budget for a geographically distributed team, there’s no funding for coaching on a team new to agile, there’s no funding for training, and so on) or there is far too much funding available and the team is motivated to invest it in wasteful activities.
• Technical risk. IT teams face technical risks as the result of combining technologies in new ways, evolving technology platforms, new technologies, and lack of experience within the team in these situations.
• Operational risk. This category of risk pertains to production-oriented issues surrounding the support and operation of a solution. Will your solution work within the existing organizational ecosystem? Does your operations team have the skills and resources to run your solution? Does your support/help desk team have the experience and knowledge required?
• Process risk. As DAD clearly shows, every technique has advantages, disadvantages, and specific situations where the technique makes sense. The implication is that there are risks taken on when you apply a technique outside of its range of applicability or your comfort zone.
• Organizational risk. IT teams face organizational risks such as dysfunctional politics, competing visions, challenges pertaining to your agile transformation challenges, reorganizations, and many others.

Recognizing that IT delivery teams must deal with risks on a regular basis DAD builds in several agile risk management strategies:

• Continuous engineering practices.  This is a collection of practices with very short feedback cycles, thereby enabling you to adjust course quickly. These practices include behavior driven development (BDD), test-driven development (TDD), continuous integration (CI), continuous deployment (CD), and many others.  All of these practices require discipline and skill to adopt.  Although these practices are technical in nature, in practice they mostly reduce business risk through shortening the cycle between a stakeholder having an initial idea and the team providing a solution which reflects that idea.
• Agile architecture practices.  The DA toolkit suggests a collection of continuous architecture strategies throughout the entire lifecycle.  These include, but are not limited to, identifying an initial technical strategy, proving the architecture early in construction, deferring architecture and design decisions to the most appropriate moment, architecture spikes, just in time (JIT) model storming, and many others.  These practices reduce technical risk.
• Risk-value lifecycle.  The DA toolkit supports several lifecycles, and hence several work prioritization strategies.  One such strategy is the Unified Process (UP)’s risk-value approach where both risk and value are considered when prioritizing work items.  The end result is that work items with higher technical risk are implemented early in the lifecycle, enabling disciplined agile teams to mitigate technical risk early in the effort.  DAD teams then follow Scrum’s value-driven approach where the work is prioritized based on its business value to the organization, which is effective at addressing some forms of business risk.  The risk-value prioritization approach results in a lower overall risk profile than just the value-driven lifecycle of Scrum.  DAD also supports lean and continuous delivery versions of the lifecycle which expand upon the risk-value strategy to consider other factors such as required release dates and team health considerations.
• An explicit Inception phase.  The fundamental purpose of the Inception phase is to help your team get going in the right direction.  Executed properly, the Inception phase helps you to reduce business risk by coming to stakeholder agreement regarding the team’s strategy, technical risk by exploring a viable technical strategy, and operational risk through planning with production-staff from the very beginning.
• Risk-based, light-weight milestones.  The DAD lifecycles include explicit milestones as part of DAD’s overall governance strategy.  These milestones motivate teams to address common issues such as formulating an agreed-to vision early in the effort (business risk), proving the architecture early (technical risk), regularly considering the viability of the effort (business risk), ensuring the team has produced sufficient functionality to justify deployment (business risk), ensuring the solution is production ready (operational risk), and ensuring that the stakeholders are delighted with the result (business risk, organizational risk).
• Development intelligence.  One of the governance strategies that the DAD process framework promotes is development intelligence (DI) where a team/project dashboard is populated with metrics generated automatically by the tools the team is using.  This provides real-time insight for the team to organize itself and potentially improve its approach.  It also provides insight into your organization’s governance effort, supplying real-time data which can enable senior management to make better decisions and thereby better support your agile teams.  Development intelligence, and it’s extension IT intelligence which addresses the entire IT portfolio and processes, helps you to reduce process risk and to a lesser extent organizational risk.
• DevOps principles and practices. DAD weaves DevOps strategies through the entire framework.  This strategies are: Including operations and support staff as explicit stakeholders who work closely with the team; the lifecycle explicitly depicts production/operations; DevOps-oriented milestones (Production Ready and Delighted Stakeholders) are included as part of the governance strategy; development practices such as instrumenting solutions for operational monitoring, continuous deployment, and installation testing (to name a few); and management practices such as deployment planning. The strategies help to reduce both operational and organizational risks.
• Sophisticated go-forward decision points.  One of the business risk management strategies promoted by Scrum and RUP is to make a “go/no-go” decision at the end of each sprint/iteration.   DAD takes this strategy further by recognizing that you have several options to consider – your team can continue with its current approach (go), it can stop (no-go), it can decide to change direction (pivot), or it can decide to experiment (run a split test).  The two new options are adopted from Lean Startup. DAD’s lean and continuous delivery lifecycles promote the idea that you make these go forward decisions when you need to, that you don’t need to wait until the end of an iteration to do so.
• Risk-oriented process goal.  The DA toolkit promotes a non-prescriptive, process goal-driven approach.  DAD explicitly includes a goal, Address Risk, the goal diagram for which is presented below in Figure 1.

Figure 1. The Address Risk process goal.

• IT risk management is built in. IT-level risk management is built into the IT Governance process blade.  Although a risk may be small for a single team, when that same risk is taken by multiple teams in parallel in aggregate the risk may be more than your organization should take on.  Hence your risk management efforts at the delivery team level must be monitored regularly and aggregate risks mitigated appropriately. The process goal diagram for IT governance is shown below in Figure 2.

Figure 2. IT-risk management is an aspect of IT Governance.

Risk management isn’t explicitly built into first generation agile methodologies like Scrum or XP, although a handful of implicit risk mitigation techniques are.  As a result you still need to develop a risk management strategy for yourself when you limit your team to these sorts of agile methods.  The DA toolkit, on the other hand, has built risk management strategies into the toolkit in a lightweight and comprehensive manner.  From the point of view of process risk, having such guidance built into your process is a low-risk and desirable approach.  For further reading about risk management, we highly suggest the list of references maintained by Glen B. Alleman.