Disciplined Agile

One of the strengths of the Disciplined Agile (DA) toolkit is that it promotes the philosophy that teams must be enterprise aware.  But what does this really mean and why is it important?  Two very good questions that I address in this posting.

For the purpose of our discussion we will focus on five levels of awareness that people may exhibit:

1. Individual awareness.  From this viewpoint it’s all about how someone can change themselves by gaining new skills, insights, experiences, and so on.
2. Team Awareness.  Here the focus is on the team can learn and improve together.  This has been a primary philosophy of the agile community for quite some time, mostly to our benefit but also to our detriment.  Solutions are developed by teams, so by promoting a greater focus on the team agilists are able to improve their overall productivity a bit.  But, if the efforts of that team aren’t well aligned with the overall goals of the organization then dysfunctions will occur.  For example, agile delivery teams that are merely team  aware may end up introducing new technologies that their operations groups aren’t willing or able to support in production.  Or they may reinvent existing functionality.  Or they may create yet another data source even though the data already exists elsewhere.  Or they may develop to their own conventions that don’t reflect the way other teams work.  Or they may learn that a certain technique or strategy works well for them, but they don’t share this learning outside of the team.
3. Departmental Awareness.  People consider the needs of their department, not just their team.  In this case developers are focused on improving the overall process, perhaps by adopting more of a DevOps mindset instead of simply a development mindset.
4. Enterprise Awareness.  People are motivated to consider the overall needs of their organization, to ensure that what they’re doing contributes positively to the goals of the organization and not just to the suboptimal goals of their team.  This is an example of the lean principle of optimizing the whole, in this case the organization, over local optimization at within just the team.
5. Community Awareness.  People consider the needs of their community, doing what they can to give back by sharing knowledge, by striving to learn themselves, and by striving to help others who might not necessarily be in their organization or even known to them.  

Of course, a given individual is very likely operating from several viewpoints at once.

Agile has done a great job of helping the IT profession refocus from individual to team awareness.  But if we want to be effective as professionals we at least need to promote the philosophy of enterprise awareness, so that we’re optimizing the work that we do for our organization.  Agile teams that are enterprise aware will work closely with enterprise professionals, such as enterprise architects and operations staff, to ensure that they are leveraging and better yet enhancing the existing infrastructure.  Their architectures will their organization’s technical roadmap and similarly the scope of their effort will reflect their organization’s business roadmap.  They will follow existing development guidelines and enhance them where appropriate.

By working in an enterprise aware manner DA teams enjoy:

• Higher levels of productivity because they are less likely to reinvent the wheel
• Quicker times to deployment/market because they have less work to do
• Higher return on investment (ROI) because they have less work to do
• Higher levels of quality through following common conventions and reuse

A stakeholder is someone who is materially impacted by the outcome of the solution.  In this regard, the stakeholder is clearly more than an end-user: A stakeholder could be a direct user, indirect user, manager of users, senior manager, operations staff member, the “gold owner” who funds the project, support (help desk) staff member, auditors, your program/portfolio manager, developers working on other systems that integrate or interact with the one under development, maintenance professionals potentially affected by the development and/or deployment of a software project.  To simplify the definition of stakeholders, we’ve adopted Outside In Software Development’s four stakeholder categories:

1. End users.  These are the people who will use your system, often to fulfill the goals of your principals.  They typically want systems which are usable and enable them to do their jobs more effectively.
2. Principals.  These are the decision makers who ultimately pay for and then put your system to use.  This includes gold owners, senior business management, and purchasers of the commercial systems.
3. Partners.  These people make the system work in production.  This includes operations staff, support staff, trainers, legal experts, installers, application hosting companies, and application developers on external systems which integrate with yours.
4. Insiders. These are members of the development team and people who provide technical and business services to your team.  This includes enterprise architects, database administrators, security experts, network experts, toolsmiths, marketing experts, and sales staff.

So why does DAD use the term stakeholder instead of customer?  Although customer is a perfectly good term, it’s very popular with agile adherents, we’ve found that many agile teams will inadvertently limit themselves to considering just end users and principals to be their customers and miss the partner stakeholders and sometimes even the insider stakeholders.  Granted, you’ll often work with some insider stakeholders as a matter of course throughout the project so it’s hard to miss these people, although both of us have seen core agile teams neglect working with their organization’s enterprise architects in the name of “having the courage to worry about tomorrow’s problem tomorrow.”  Our experience is that although the term stakeholder is a bit more formal than the term customer, in practice it seems to lead agile teams to a more mature strategy.

One challenge with the term stakeholder, which customer also suffers from, is that it reinforces the “them vs. us” mentality prevalent in many IT departments.  When we use terms such as “the stakeholders” or “the customers” or “the business” they imply that we see ourselves as a group set apart from them.  The reality is that it isn’t the business, it’s our business, that we’re all in this together.  It’s important to be careful with terminology.

Elizabeth Woodward, co-author of one of my favorite books on scaling agile, A Practical Guide to Distributed Scrum, recently posted a video on Youtube entitled Disciplined Agile Delivery – The Shirt, the Summary .  I worked with Elizabeth at IBM when she led IBM’s internal agile educational support effort and actively worked with teams to help them to adopt and tailor disciplined agile strategies.  As you can read in her own book, she has deep expertise in successfully applying agile at scale.

In the video, which I had no idea that she was going to create, she shares her thoughts about why you should consider DAD.  She describes the value of the explicit inclusion of Inception activities, extending Scrum to describe a truly hybrid approach, and the explicit inclusion of transition activities.  The video is a little less than six minutes in length and I suspect you’ll find it a good use of your time.

As the title implies, she also has a few nice things to say about the “fabulous” DAD shirt we sent her.  Luckily, due to training from my wife, The Lovely Beverley, I knew enough to order a woman’s version for her.  Context counts!  

I was recently asked the question “What happens when the Product Owner and the Architecture Owner don’t agree?” and realized this is an issue for everyone on DAD teams in general.  Here’s my advice:

1. Talk it out.  People aren’t always going to agree, and that can often be a very good thing.  So talk it out amongst yourselves and explore why you don’t agree on an issue.  Very often someone else has a different point of view that you weren’t aware of, hence the need for a discussion.
2. Recognize that people have certain rights and responsibilities.  One of the reasons why DAD defines rights and responsibilities for various roles, which we adapted from source methods such as Scrum and Agile Modeling, is to help distinguish the decision rights of people in those roles.  For example Product Owners have the right to prioritize the work but do not have the right to dictate technical decisions.  Similarly, the Architecture Owner is responsible for guiding the team through technical decisions but does not have the right to set work priority.  An implication is that although the AO might not agree with some of the prioritization decisions being made by the PO they still need to respect those decisions.
3. Talk it out.  It’s better to talk an issue through and communicate the reasons behind a decision than to simply dictate it.

The Situation Context Framework (SCF), an evolution of the Software Development Context Framework (SDCF), defines the contextual factors to consider when selecting and tailoring a situation-dependent way of working (WoW).  The SCF is used to provide context for making decisions about how to organize your WoW to be fit-for-purpose.  Figure 1 overviews how several selection factors drive the initial choice and tailoring of your team high-level WoW, in particular your choice of lifecycle.  Of course initial selection is just the first step, you will also need to tailor your detailed choices to reflect the situation that you face - these decisions are driven by the complexity factors that you face.

Figure 1. Context factors for selection and tailoring your way of working (WoW).

Selecting an Initial WoW 

When you initiate a team you need to identify key aspects of your WoW, in particular:

1. Your team organization. When it comes to team organization you have several issues to consider.  Will the team be composed mostly of specialists such as business analysts, user experience (UX) designers, implementers and so on or will team members be more along the lines of T-skilled generalizing specialists?   How large will the team need to be?  Where will you find these people?   Will they be located in the same place or spread out?  Will they work for a single organization or several?  The choices you make will be driven by the situation that you face.
2. How you will work together. Similarly you have several process-related issues to consider.  What paradigm is most appropriate?  For example will you take an agile approach?  A lean approach?  A traditional approach?  A hybrid approach?  Will your team be able to follow a light, goal-driven process or a prescriptive one?  Will your process be constrained by compliance to frameworks such as CMMI or ISO standards?
3. What tools you’re going to use. With respect to tooling there is a myriad of options and it seems as if everyone has an opinion as to which tools are best.  However, our experience is that there are several key issues to consider when choosing tools.  Will you adopt open source tools, commercial tools, or a combination thereof?  Will your tools be integrated or stand alone?  Do you prefer to obtain tools from a single source whenever possible, with the potential for better integration and support, or will you strive for best of breed tools regardless of vendor?  Will you host your own tool environment or will it be hosted externally via a SAAS-style approach?  If hosted externally, where will your intellectual property (IP), such as source code, be hosted?

The choices that you make initially will change over time as you learn and as your situation evolves, the point is that you will make some broad choices at first to get going.

The Selection Factors

The decisions about your initial WoW will be driven by factors such as the skill and culture of the people who will potentially be on the team, your organizational culture and policies, the nature of the problem being addressed, and business constraints such as time to market and budget. Figure 2 overviews these selection factors, indicating the range of the extremes for each one - On the left-hand side is the simple extreme and on the right-hand side the challenging extreme. .

Figure 2. Selection factors.

The selection factors are:

1. Team member skills.  The people on a team must have the skills, or must gain the skills, that befit the role they play on that team.  For example, developers on an agile software team may need to have test-driven-development (TDD) skills, people-oriented collaboration/communication skills, continuous integration (CI) skills, model storming skills, team-based planning skills, and so on.  Developers on serial/traditional teams may be more focused on programming skills for a specific technology platform.
2. Team culture.  People who are collaborative and team-focused in nature are better suited for agile/lean environments whereas people who like to work alone are better suited for traditional approaches. Similarly people who are open and flexible in their approach are better suited for agile or lean strategies.
3. Organizational culture.  Your organization’s culture may vary from that of the team you are putting together, something that is particularly true when you are first learning about new ways of working.  An organizational culture that is very flexible and collaborative will mean that it is easier to take an agile or lean approach, wherease a more rigid, command-and-control culture will make it difficult to do so.
4. Nature of the problem.  Although some people want to believe that certain types of problem can only be solved in one manner that doesn’t seem to be the case in practice.  For example, it’s possible to take an agile or a traditional approach to data warehousing projects, to marketing projects, and to procuring services from an external partner.  Our experience is that the real issue is how decomposable the potential solution is.  For example, it’s possible to decompose a data warehouse into many small releases if you choose.  Same thing can be said of the development and execution of a marketing campaign.  But, it isn’t very easy to decompose an airplane into several working parts.  Either the airplane is complete or it isn’t.  Yes, it’s still possible to apply agile techniques to the building of the airplane, and very likely to it's design as well, but the airplane team will never be as agile as a software development team due to the physical and regulatory constraints that they face.
5. Business constraints.  The way that the business constrains the endeavor, such as insisting on a certain (always agressive) delivery date, an approach to managing the budget (often a fixed price/bid), and how available business people will be throughout the project certainly has an affect on the process you adopt and the type of people that you include on the team.  It may even influence what tools you use, particularly those for communication and collaboration.

The Complexity Factors Pertinent for Choosing Your WoW

The complexity factors of the SCF affect your decisions when choosing techniques/practices when you choose and evolve your WoW. Figure 3 explores these complexity factors, indicating the range of each factor.  On the left-hand side is the simple extreme and on the right-hand side the challenging extreme.  

Figure 3. Complexity factors.

Let’s examine each scaling factor one at a time:

1. Team size.  Teams can range in size from two people to twenty to two hundred or more.  Larger teams are typically formed to address more complex problems, and as a result large teams take on the challenges of greater domain complexity and/or greater technical complexity as described below.  Team size tends to directly affect how you organize the team and how you coordinate within the team.  For example, a large agile team of 600 will be organized into subteams and a leadership team will be required for coordination, something we capture in DA's Program life cycle.  A team of 50 will also be organized into subteams, although coordination will likely be simpler and possibly handled by a daily coordination meeting of representatives from each subteam (a techniques referred to as a Scrum of Scrums).  It is fairly straightforward to coordinate the activities of a team of 10 people.
2. Geographic distribution.  Agile teams may be co-located, with the team members and key stakeholders in the same room, they may be on the same floor in a single building, on multiple floors, some may work in different buildings, some may work from home, and some may even work in different countries.  A popular misconception is that agile teams need to be co-located, a misconception that I have shown via several surveys over the years to be false.  Granted, it’s a very good idea to get people working as closely as possible, but it doesn’t happen as often as we’d like.  Similar to large teams, coordination of team members throughout the project become more difficult and as a result more sophisticated coordination is required.  A greater investment in initial modeling and planning, but not much more, is required during Inception to mitigate the communication risks associated with distribution.  To increase chance of project success you will need to fly people around at key points in the project, something many organizations are loathe to do because it’s easy to measure travel costs but difficult to quantify the benefit of face-to-face collaboration.  Please read Geographically Distributed Agile Teams for a more detailed discussion.
3. Organizational distribution.  This refers to the concept of involving people from several organizations on the project.  The easiest situation to be in is to have all of your team members from the same group/division within a single organization, often the situation of a startup company or new product team within an enterprise.  It’s a little harder when people from several groups are involved.  Hiring contractors adds to the complexity.  Outsourcing a portion of the work to an external service provider is harder yet.  Partnering with several vendors even harder.  Outsourcing to one or more service providers with a very different culture than your own harder yet.  Organizationally distributed projects tend to take on the challenges associated with large teams and geographically distributed teams.  When outsourcing is involved they take on the risks associated with procurement and then the governance of the outsourced effort.
4. Skill availability. Your team needs the right people with the right skills to fulfill the outcomes that you've taken on.  At the ideal end of the spectrum you have skilled people "sitting on the bench" waiting to get going, at the other end it may take many months and potentially lots of money to build the team that you need. The availability of skilled people, or at least people with the ability to quickly gain the skills that they need, is a driver of organization distribution.  When your immediate organization doesn't have easy access to the required skills you may need to start partnering with other groups or even external organizations to gain them.
5. Compliance.  There are two forms of compliance.  Generally the simpler form of compliance is self-imposed, perhaps your organization chooses to be CMMI or ITIL compliant.  The second, and potentially harder, form of compliance is regulatory.  A team may need to conform to financial regulations, privacy regulations, or even existential/life-critical regulations.  Although every regulation has different requirements, from a process point of view they typically require extra documentation (but keep it light), reviews (keep them streamlined), and sometimes a documented process.
6. Domain complexity.  The complexity of the domain, or the problem space, being tackled by a team can vary widely.  An informational website site, such as this one, is fairly straightforward.  An e-commerce site is more difficult.  An air traffic control system even more difficult.  The greater the domain complexity that you face the more you want to invest in up-front modeling and planning.  Not much more, mind you, but still more.  Similarly as domain complexity rises it motivates greater sophistication in your agile testing strategy.  As domain complexity increases it puts a greater burden on your Product Owner, requiring more sophsticated agile modeling skills and potentially the support of agile business analysts.
7. Solution complexity.  Disicplined agile teams will face varying levels of solution complexity.  On the simple end of the spectrum you’ll have the development of a brand-new, stand-alone solutions built with new technologies.  Things get more difficult if you need to take advantage of existing assets, including software, data sources, or business services.  Things get more difficult if you need to support several technology platforms.  Things are more difficult yet if you need to refactor existing infrastructure (including legacy data sources).  As with domain complexity, the greater the solution complexity the greater the need for a bit more up-front modeling and more sophisticated testing throughout the lifecycle.  Greater techncial complexity puts a burden on your Architecture Owner, requiring great agile architecture and agile design skills of them.

History of the SCF

Where do these ideas come from?  The primary source is something called the Agile Scaling Model (ASM) which I led the development of in 2008-2009 while working for IBM.  In parallel to my work on the ASM Philippe Kruchten was working on something he calls “situational agility”, the heart of which was eight (8) factors often referred to as the “Octopus model”.  In the Autumn of 2012 Mark Lines and I began thinking about how to combine and evolve these two frameworks into one, something we originally called the Process Context Framework (PCF).  We moved away from that name because the strategy was clearly applicable to more than just software process, hence we adopted the name Software Development Context Framework (SDCF) which is inclusive of people, process, and tools.  Then of course, over the years, we applied this to far more than just software development, so we evolved this to become the Situation Context Framework (SCF) in 2020.

Related Reading

• Guided Continuous Improvement (GCI)
• The Evolve Way of Working (WoW) process blade