Disciplined Agile

Although we strive to avoid functional dependencies between requirements, the fact is that they occur in practice.  They certainly occur between requirements that are being addressed by a single team and they will even occur between requirements being addressed by different development teams.  This is particularly true between the subteams within a program (a large development team organized into a “team of teams”).

The following diagram depicts a simple situation where there is an agile program organized into five subteams (A thru E).  The way you read the diagram is that the arrows between requirements represent functional dependencies.  In this case the requirement fourth from the top on A’s backlog (A4)  depends on the one that is second from the top (A2) and on B3 (the requirement third from the top on team B’s backlog).  B3 in turn depends on C7, which depends on D3.  D3 depends on E4, which depends on D10, which depends on C12, which depends on D14.  There could very well be dependencies between other requirements, but for this example we’re only worried about the dependencies originating from A4.

Figure 1: Example of functional dependencies between requirements.

Where Do Functional Dependencies Come From?

Functional dependencies occur between requirements for several reasons:

1. End-user driven.  Functional dependencies occur naturally in the business domain as the result of end user activities.  For example, when a customer opens a bank account there is a functional dependency between the customer and bank account business entities.  Furthermore, functionality such as being able to withdraw from a bank account depends on their being the ability to open a bank account for a customer to begin with.
2. Requirements decomposition. When a large requirement is decomposed into smaller ones there are dependencies from the original large requirement to the smaller sub-requirements.  An example of this is decomposing an epic, a large story, into several smaller stories.
3. Technology driven. Some teams will choose to identify requirements for a given platform, subsystem, or architectural layer.  For example, you may identify requirements for systems of engagement, such as applications running on mobile devices, and for systems of record, such as a back-end ERP system.  A requirement for a mobile application may have a dependency on a requirement for a backend system to provide certain behaviors (such as the ability to create, read, update, and delete data).  An example of requirements dependencies driven by architectural layering would be that you may choose to identify user interface (UI) requirements, business rules, and data requirements (via data models perhaps).  A UI requirement depends on the implementation of a business rule which in turn depends on several data-oriented requirements.

Of the three reasons for why functional dependencies exist, the first two are clearly within the purview of a Product Owner (PO).  The third one, technology driven dependencies, can be trickier because many POs will not be familiar with the underlying technologies.  This is one reason why Disciplined Agile Delivery (DAD)’s Architecture Owner (AO) role is so important.  The AO and the PO on a disciplined agile team work very closely together.  AOs will help POs to better understand the implications of the technologies being used as well as to understand potential implications of the technologies (including technology-driven functional dependencies).  These sorts of discussions will occur throughout the lifecycle, although they are particularly important during initial release planning during Inception and during iteration planning and look-ahead planning throughout Construction.

How do You Resolve Functional Dependencies?

Functional dependencies are addressed via three basic strategies:

1. Reprioritize one or both of the requirements.  When requirement X depends on requirement Y you ideally want to implement Y before, or at least in parallel, to implementing X.  The advantage of reprioritization is that it requires the least amount of work by the development team.  The disadvantage is that when a requirement is reprioritized in this manner the team is no longer working on the highest priority functionality by business value, potentially decreasing the return on investment (ROI) provided by the team.
2. Mock out the missing functionality until it is available.  Requirement X depends on requirement Y and Y will not be available in time (e.g. X is being worked on right now and Y will be developed in a future iteration).  In this case the development team will implement X to the best of their ability, but will mock out (simulate or stub out) the functionality of Y until it is available.  The advantage of this approach is that it is now possible to demo X so as to get feedback about it.  The disadvantages are that your solution isn’t shippable until the mocked out functionality is implemented (or removed) and that there is the additional work to be done to mock it out.
3. Rework the requirements to remove the dependency.  If X depends on Y, then one solution might be to refactor X into X1 and X2, where X2 has the functionality dependent on Y but X1 has no dependency.  X1 would be implemented now, and X2 at the same time or after Y is implemented.  For example, a new screen has a dependency on data services being available.  The screen includes five fields, four of which already have data services available but one of which is brand new.  In this case X1 would be to do the work to implement the screen with the four fields and X2 would be the requirement to add the new field once the data was available on the back end. Another solution would be to not add the new data field at all, something you would need to discuss with your stakeholders.

Within an agile environment, functional dependencies are managed by Product Owners.  When the dependencies are between requirements, or more accurately work items, that are being addressed by a single agile team this is fairly straightforward because it’s within the purview of the single product owner.

Now let’s consider how a Product Owner team would manage requirements dependencies within a program, as in the diagram above.  In this case let’s assume that each team has it’s own product owner (PO) whom we’ll refer to as PO-A, PO-B, and so on.  There are three scenarios to consider when managing requirements dependencies:

1. Within the same sub-team.  An example of this is A4 depends on A2. This is straightforward as a single person, in this case PO-A, is responsible for managing this dependency.
2. On previously developed functionality.  An example of this is C7 depends on D3.  This should also be straightforward as D3 was implemented in iteration N so it should be available when C7 is implemented during iteration N+1.
3. On future functionality.  An example of this is B3 depends on C7.  The problem is that we want to implement B3 during iteration N but we currently plan to implement C7 during iteration N+1.  The two product owners, PO-A and PO-B, will need to work together to determine a strategy for resolving the dependency.  They’ll do this via a combination of the strategies described earlier.  They may also need to work closely with the Chief Product Owner to ensure that their reprioritization choices reflect the overall needs of the program.

How would the entire requirements dependency map for A4, as depicted in the diagram above, be resolved?  It depends on what the product owner team decides.  At the present moment the overall requirements dependency chain is to be implemented during iterations N through N+3.  This may in fact be acceptable and the product owners will decide to live with the impact of mocking out functionality until it is available.  Or they may decide to reprioritize the functionality so that it is implemented in a more effective order (perhaps during iteration N+1).  The point is that the product owners will engage in negotiation amongst themselves to determine the best order in which the sub teams will implement the functionality.

How do You Manage Functional Dependencies?

Functional dependencies are managed by the Product Owner, or in the case of a program, the Product Owner team.  The goal is to do just enough work to maintain the dependencies and no more.  When you do not sufficiently maintain the dependencies, perhaps you forget to record that a sub-requirement was created as the result of decomposing a larger parent requirement, then it becomes difficult to ensure that a requirement is properly implemented. When you invest too much effort into maintaining functional dependencies any extra effort beyond the point of sufficiency is a waste.  In short, your dependency map should be just barely good enough (JBGE) for the situation you find yourself in.

There are several options available to you for maintaining a functional dependencies:

1. Physical dependency map.  With this strategy requirements, such as user stories or features, are captured on paper (typically via index cards or sticky notes) and placed on a whiteboard, corkboard, or table.  On physical boards dependencies can be indicated via physical placement, for example the cards capturing the sub-requirements of a large requirement are placed immediately to the right of the large requirement card.  On a corkboard strings representing requirements could be placed from one card to another and on a whiteboard lines could be drawn to represent the dependencies.  Or IDs could be given to each requirement and any dependencies simply captured on the cards via writing down the appropriate IDs. An example of a physical map include user story maps, see Figure 2 below, that indicate the epic or theme that a story is part of (this is a rudimentary form of dependency mapping).  Another example includes a program plan board, an idea promoted by SAFe, where requirements are mapped to iterations/sprints (columns on the board), to implementation teams (rows on the board), and dependencies indicated via strings or lines.
2. Simple electronic tool.  Dependencies can be managed using tools such as spreadsheets, word processors, or wikis.
3. Backlog/work item management tools.  This includes products such as Trello, Atlassian’s JIRA, Rally and VersionOne.  Some of these tools will have native support for managing dependencies where others do not (if the solution is to add a text reference into a freeform notes field then that’s not native support).
4. Requirements management tools.  This includes products such as Blueprint, Enterprise Architect, or DOORS NG.  Be aware that some of these tools will have native, and more importantly effective, support for agile requirements artifacts such as user stories and acceptance criteria.  Sometimes they will not.

Figure 2. Example of a simple story map.

Which approach should you take?  Our advice is always to keep it simple and use physical tools if your situation permits.  Having said that, there are advantages and disadvantages to each strategy:

• Physical dependency maps work well when teams are geographically close (working on the same floor or at least nearby floors in the same building).  When you find yourself with a lot of dependencies to manage, or in a regulatory environment when traceability is mandated, or your team is geographically distributed (even if you just have a few people working from home), you’ll find that you’ll want to consider using electronic tools.
• Simple electronic tools work well when the team is small to medium sized (say less than 30 people), when the team is geographically distributed in some way, and the dependencies aren’t very complex.
• Backlog management tools are an effective option if you’re already using them for other reasons (such as managing your work), they natively support dependency mapping, and when physical dependency maps haven’t worked out for you.
• Requirements management tools are appropriate when you find yourself in complex situations, often at scale.  This includes geographically distributed teams, complex domains, and regulatory situations.

Parting Thoughts

Earlier we said that the diagram represents a “simple” situation.  It is simple in that all five teams are following the same sort of lifecycle, in this case the basic/agile DAD lifecycle.  Furthermore the velocities of the teams are roughly the same, which we did for the convenience of the example.  Usually the team velocities are very different, due to a combination of different team sizes and different levels of productivity.  In a future blog posting we’ll discuss the challenges that arrive when the subteams are following different lifecycles (for example a lean or continuous delivery lifecycle or even a waterfall lifecycle).

Several of the strategies described in the blog posting were first identified in Complex Requirements on an Agile Project (Dr. Dobbs Journal, October 2008).

Large solution delivery teams, let’s say fifty or more people, are often referred to as programs (programmes in UK English). Such teams are often organized into teams of teams, as depicted in the diagram below.  Each of the sub teams will work on their part of the overall solution and to do so effectively there needs to be coordination between the sub teams.  On large agile teams this coordination often proves to be complex, which is why a leadership team is introduced.  This leadership team coordinates:

• Requirements. Requirements are managed by the Product Management (also called a Product Owner) team.  This team is made up of the Product Owners from each sub team.  They will meet as needed, typically several times a week. This group is the focus of this blog posting.
• Technical concerns. The Architecture (or Architecture Owner) team, comprised of the Architecture Owners from each sub team, is responsible for identifying and then governing the architectural strategy for the program.  Activities of this group include negotiating changes to the architecture vision over time, resolving disputes about technical issues between sub teams, and sharing technical learnings across sub teams.  It is common for this team to meet weekly with ad-hoc discussions occurring on an as-needed basis.
• Management concerns.  Management concerns, such as members of different teams not getting along, transfers of people between teams, and schedule dependencies will be coordinated by the Team Leads from the sub teams.  This team is often called the Product Delivery team or simply the Management team (yuck).  As with the Product Management and Architecture teams this team will meet regularly as appropriate.
• Itself.  This is the responsibility of the Program Manager. This person may be a Team Lead on one of the sub teams, although more often than not fulfilling this role proves to be a full time job.  The Program Manager will guide the overall program team, ensuring that the three leadership sub teams are working together effectively and that they are meeting to coordinate their own activities as appropriate (and typically on different schedules).

Product Management/Ownership Team Organization

The Product Owner in each sub team is a member of the Product Owner team for the program, as depicted in the following diagram.  Individual Product Owners will typically spend 80-90% of their time on activities that are directly related to supporting their sub teams and the rest of the time to requirements management activities at the program level.  The Product Owner team is lead by a Chief Product Owner (CPO).  The CPO may be a PO on a delivery team, this is common on small programs, although for larger programs the responsibility of leading the Product Owner team will prove to be full time work.  In organizations with a strong Product Management culture, the Chief Product Owner may be a senior Product Manager.

This team is responsible for requirements management activities within the program.  This includes:

1. Identifying the initial scope of the program.  The PO team will perform just enough initial requirements modelling, with active stakeholder participation where possible, to identify the initial scope of the program.  This scope is very likely to evolve over time, but for now the goal is to explore the scope sufficiently to get the program headed in the right direction.  See the process goal Explore Initial Scope for more details.
2. Ongoing requirements elicitation.  A primary job responsibility of anyone in the Product Owner role is to elicit and explore stakeholder requirements.  In the case of a program the entire PO team must coordinate their requirements elicitation efforts.
3. Assigning requirements to sub teams.  As new requirements are identified the PO team will collaborate to identify the appropriate sub team to perform the work and then assign the work to that team.
4. Managing requirements dependencies.  There are always dependencies between requirements, and these dependencies should be managed by the appropriate Product Owners.  For example, if a requirement (R1) assigned to sub team A depends on a requirement (R2) assigned to sub team B then ideally R2 should be implemented either before or at the same time as R1.  Otherwise the people implementing R1 will need to mock/stub out the missing functionality until it becomes available.  Read Managing Requirements Dependencies Between Agile Teams  for more details.
5. Developing a product roadmap.  The PO team is responsible for developing a product roadmap for the program which lays out a high-level business direction for the product. This roadmap should reflect your organization’s overall business roadmap, if you have one.

The Product Owner team will meet as often as they need to.  We’ve seen some PO teams meet on a daily basis for 30 minutes each to manage requirements between sub teams.  We’ve also seen PO teams that meet weekly for two hours to do this work.  The important thing is that they self organize to determine what works best for them.

The Product Owner team may include business analysts (an example of a specialist role in DAD) who supports the POs in working with stakeholders to understand their requirements.  This is particularly important whenever the team is addressing significant domain complexity or whenever stakeholders are geographically dispersed.

Tailoring Considerations

In medium-sized enterprises this Product Owner team approach may be applied to your entire IT department.  In this case the focus of the PO team is that of your entire portfolio of ongoing IT solution delivery efforts and not just a single program of interdependent teams.

In large enterprises the Product Owner team for a program may be part of a larger Product Management team for the entire organization.  More on this in a future blog posting.

With some basic agile training and help from an agile coach it can be relatively straightforward to enable several teams to be able to produce increments of consumable software every two weeks.  Unfortunately many organizations stop there, believing that they are “now agile”.

For enterprise agile adoptions starting a few agile teams is the easy part and is just the beginning of your agile transformation.  Proving the benefits and sustaining the change is significantly more challenging.

For illustration purposes, let’s assume that over a six month period we have conducted a series of Disciplined Agile workshops and kickstarted twelve teams.  The teams have separate product owners with their own work item lists.  Some of the teams use the DAD basic Scrum-based lifecycle while others use the DAD Lean lifecycle.  The Scrum teams adapt their lifecycle for their context with the simpler projects having a one week Inception phase while the more complex projects use a two week Inception.  For the Construction phase novice Scrum teams use two week iterations while the more advanced teams chose one week iterations.  The teams vary in size from four to twelve team members.  By rolling out the teams in an incremental fashion, with some coaching the teams have learned the key DAD practices for being effective on both the Scrum and Lean teams.  Word is spreading that the teams are impressing stakeholders with regular demonstrations of new functionality.

However, after the honeymoon period is over, people start to ask some interesting questions such as:

• What are the limits of self-organization?  I understand that teams are free to customize their own processes but isn’t some consistency good across teams?
• What are the key milestones for each team?  What is the release schedule for each team?  Are the teams on track for delivering the solution consistent with their Inception vision?  How do I see this information?
• How do I know which teams have the greatest risks outstanding?
• What is our product roadmap strategy across teams?
• How do we measure the effectiveness of one team vs. another?
• How do we measure the effectiveness of individual team members?
• What is the new career path for agile team members?
• How to we adjust compensation plans to encourage effective team behavior and reward individual contributions?
• Has our quality assurance group adapted for agile to have the appropriate mix of embedded vs independent testers?
• Do we have less technical debt than before agile?  How do I know if our quality is improving?
• How do I know that teams are effectively engaging with enterprise authorities such as architecture, data, and quality assurance?
• How can management use agile and lean principles themselves?

These are all fair questions.  For your agile adoption to be effective and sustainable you need a strategy to address all of these issues.  The Disciplined Agile Manifesto adds a principle to the Agile Manifesto regarding the need to adapt your organizational ecosystem to be effective for enterprise agile adoptions:

The need for good governance doesn’t go away with agile.  Your stakeholders deserve the right to gauge the health of their investments in your agile initiatives just like any other project.  There are indeed answers and various strategies for all of the above questions which will vary depending on the context of your situation.  Like it or not, the reality is that effective and sustainable agile transformations can take several years in order to achieve the level of capability and maturity that you expect.  A transformation is a journey, not a destination.  Make sure that your agile coaches have answers to the questions above and know what to do after your Scrum honeymoon period is over.  We will outline some DAD strategies for the questions above in future posts.

For a more detailed discussion of how DAD extends Scrum, please read the whitepaper Going Beyond Scrum.

One of the key philosophies of the Disciplined Agile (DA) toolkit is that it presents software development teams with choices and guides them through making the right choices given the situation they face.  In other words, it helps them to truly own their process.  Part of doing so is to choose the software delivery lifecycle (SDLC) that is the best fit for their context.  In this blog posting we overview the DAD Exploratory lifecycle which is based in part on Lean Startup strategies.

This lifecycle can be applied in two ways:

1. As a replacement of the Inception phase of other lifecycles.  In the Inception phase we invest a short yet sufficient amount of time and effort to validate that the initiative being considered makes sense and to gain agreement on the stakeholders’ vision.  In a situation where the actual need and value of what is being proposed is in question this approach is a very good way to determine the true market need before scaling up the initiative and moving into the Construction phase.
2. As the implementation approach in the Construction phase.  After applying the Exploratory approach to validate your vision, a decision needs to be made regarding which of the four DAD lifecycles to apply as we move into Construction. For instance, you may choose to use DAD’s Scrum-based basic agile lifecycle if there is sufficient confidence from the learnings in the Inception phase regarding the viability of the vision.  However, if there remains some uncertainty regarding the feature set to be delivered it may make more sense to continue using the Exploratory lifecycle to build out the product in Construction.

The following diagram overviews the DAD Exploratory lifecycle.  This lifecycle is followed by agile or lean teams that find themselves in startup or research situations where their stakeholders have an idea for a new product but they do yet understand what is actually needed by their user base.  As a result they need to quickly explore what the market wants via a series of quick learning experiments.

Now let’s describe how the Exploratory lifecycle works.  There are six activities to this lifecycle:

1. Envision.  Your team will explore the idea and identify a potential implementation strategy for implementing it.  This could be as simple as getting a few people together in a room to model storm both the business vision and your technical options on whiteboard and paper.  You want to do just enough thinking to identify a viable hypothesis for what your customers actually want.  This hypothesis needs to be testable, which implies that you need to identify how you are going to measure the effectiveness of the new functionality that you produce.
2. Build a little.  Your team should invest just enough effort to build a solution that tests the hypothesis.  In lean parlance you want to build what’s known as a minimally viable product (MVP).  The amount of effort will vary, from several days to several weeks – your goal is to make something available very quickly so that you can test your hypothesis.
3. Deploy.  Once your current solution is ready it is deployed into an environment where you can test your hypothesis. This deployment may be to a subset of your customers, in many ways what used to be called an “alpha” or “beta” release, so that you can determine whether the solution is of interest to them.
4. Observe & measure.  Once the solution is available in production you want to determine what aspects of it, if any, are of interest to your user base.  To do this you will need to instrument your solution so that it logs data regarding important events within it.  For example, you may decide to record when a screen/page is accessed, when a sale occurs, when certain business functions are invoked, and so on.  The idea is that you want to understand which functionality end users find useful, which functionality leads to customer retention, which functionality leads to greater sales, … whatever is important to you.  Generation of this data enables you to monitor, to observe and measure, how well the new functionality is received by your user base.  This in turn allows you to make a fact-based go-forward decision.  If the functionality is well received then you may choose to continue with the existing strategy and add more functionality.  Or your strategy may be so successful that you decide that you’re ready to productize the development of this solution. If the functionality wasn’t well received your team might choose to pivot and continue in another direction or even give up completely.
5. Cancel.  Sometimes you discover that the product idea isn’t going to work out after all.  In fact, this is particularly common in research and development (R&D) environments as well as start ups.  The advantage is that if an idea is going to fail, then it is better that you learn that it’s a failure quickly so that you can devote your energies into other strategies.
6. Productize.  After several iterations of building a little, deploying, and then observing & measuring that you’ve identifying a product that will be successful in the marketplace (or in the case of internal application development successful with your user base).  As described above, although you may choose to continue following this lifecycle, a common decision is for the team to adopt one of the other DAD lifecycles – such as the Scrum-based agile lifecycle, the Kanban-based Lean lifecycle, or the Continuous Delivery lifecycle – and effectively treat the time they spent following this lifecycle as their Inception phase.

To summarize, the DAD process framework takes a flexible, non-prescriptive approach to software-based solution delivery.  As a result of this philosophy DAD supports several development lifecycles, one of which is the Lean-Startup-based Exploratory lifecycle described in this posting.  This lifecycle is typically followed in situations where you are unsure of what your user base wants, and sometimes even when you are unsure of who your user base (your customers) will even be.

We have just published a Disciplined Agile Manifesto, an extension to the original Agile Manifesto.

Why have we done this? Since 2001 we’ve applied the ideas captured in the Agile Manifesto and have learned from our experiences doing so.  What we’ve learned has motivated us to suggest changes to the manifesto to reflect the enterprise situations in which we have applied agile and lean strategies.  Because the original authors of the Agile Manifesto have made it clear that they intend to keep their Manifesto static we have decided to move forward on our own with this extension.

We believe that the changes we’re suggesting are straightforward:

1. Where the original manifesto focused on software development, a term which too many people have understood to mean only software development, the DA toolkit suggests that we should focus on solution delivery.  In short, we prefer solutions over software.
2. Where the original focused on customers, a word that for too many people appears to imply only end users, the DA toolkit suggests that it focus on the full range of stakeholders instead.  We prefer stakeholders over customers.
3. Where the original manifesto talked about projects, we believe it is more accurate to talk about teams.  As a result we replaced the word project with team throughout the principles.
4. Where the original manifesto focused on development teams, the DA toolkit suggests that the overall IT ecosystem and its improvement be explicitly taken into consideration.
5. The original manifesto focused on the understanding of, and observations about, software development at the time.  Since then there has been some very interesting work done within the lean community since then (and to be fair there was very interesting work done within that community long before the Agile Manifesto was written).  This manifesto incorporates lean principles, in particular considering the whole, visualizing workflow, and minimizing work in progress (WIP).

For earlier versions of the Disciplined Agile Manifesto, see Reworking the Agile Manifesto on IBM Developerworks and the book Choose Your WoW!