Disciplined Agile

This blog posting has been replaced by Enterprise Architecture Team Structures.

Organizations face an increasingly competitive marketplace.  To be competitive, organizations must be able to react to environmental changes and bring new offerings to market quickly.  This requires an adaptive approach to IT, which in turn requires a flexible and malleable approach to enterprise architecture.  To succeed in this new organizational climate enterprise architects require a new mindset, and new ways of working, that reflect the realities that they now face.

In this blog posting we describe a disciplined agile mindset for enterprise architects.  The following terms describe this mindset:

• Collaborative.  First and foremost, disciplined agile enterprise architects (EAs) collaborate closely with their stakeholders.  These stakeholders include business executives, IT executives, IT operations staff, and of course IT solution delivery teams.  Disciplined agile EAs will spend the majority of their time working with stakeholders.
• Learning oriented.  Every day, disciplined agile EAs strive to learn more about the people they are working with, about the business, about the business environment, about the technology and its application, and about the process they are following.
• Sharing. Disciplined agile EAs, and disciplined agilists in general, constantly look for opportunities to share their skills and knowledge with others.  An important part of their job is to help those around them to get better.
• Business focused.  Disciplined agile EAs must have an understanding and appreciation of the business and the way that it operates to effectively interact with, and support, business leaders.
• Multidisciplinary.  Enterprise architects will often start their architecture career as a solution architect, or a data architect, or a business architect, or some other architecture specialty.  Although these are good starting points, enterprise architects need to consider a wider range of issues than those focused on by each of these specialties.  Just as enterprise architecture frameworks such as TOGAF, Zachman, DODAF and others all take a multi-view approach, disciplined agile EAs must have the skills, or be in the process of gaining them, to address a wide range of views.
• Evolutionary. Disciplined agile EAs work in an evolutionary manner.  As you see in the following diagram, when your enterprise architecture team is first formed it may invest a bit of time to perform some initial architecture envisioning.  The team should come to an agreement around their overall vision and get some of the fundamentals captured.  Then the EAs will work with business stakeholders to understand and guide their vision for their lines of business, in particular how they can leverage technology more effectively to provide greater value to the organization.  The EAs will also work closely with IT delivery teams, often taking on the role of architecture owner on the team, so as to help guide and coach the team in architectural skills and concepts. The EAs will also meet on a regular basis, perhaps weekly for a couple of hours, to share what they’ve learned amongst themselves to evolve the architecture assets based on their learnings.  More on this evolutionary approach in a future blog posting.

• Practical.  Disciplined agile EAs are not afraid to get their hands dirty, which is why they will work collaboratively with their stakeholders.  They recognize that for developers, who are important stakeholders of their enterprise architecture efforts, that the proof is in the code.  As a result disciplined agile EAs will produce working code examples as the primary component of their reference architectures.  They realize that developers are much more likely to work with (and then copy) high-quality working code than they are to read a detailed white paper or detailed model extolling the virtues of some architectural concept.
• Pragmatic. Disciplined agile EAs know when to trade-off short and long-term gains and more importantly can help guide their stakeholders through these decisions while coaching them on the implications of what they’re doing.
• Improvement oriented.  Disciplined agile EAs are always looking for ways to streamline the processes that they run into.
• Reuse.  Disciplined agile EAs promote a reuse mindset with both their business stakeholders and with IT delivery teams.  To achieve reuse on a large scale within your organization there needs to be a guiding vision, and that vision is provided by your enterprise architects.  More on this in coming blog postings.
• Technical debt savvy.  Disciplined agile EAs should be a driving force within your organization for addressing technical debt.  They will guide and coach IT delivery teams on both avoiding new debt and on removing old debt.  In parallel they will work with business leaders to understand the implications of technical debt and help guide them in supporting IT delivery teams in addressing it.
• Technical.  And finally yes, disciplined agile EAs must have a technical background so that they understand the implications of the technologies in use (either now or in the future) within their organization.  As noted earlier, a critical strategy for keeping their technical skills fresh is to collaborate with IT delivery teams on a regular basis.

All the features on this list should sound like common sense – that’s because they are.  It’s probable that you’ve worked with enterprise architects in the past whose mindset exhibited many of these features, but did they exhibit all of them?  Likely not.  No matter how good you are, there is always room for improvement.

Related Readings

• The Enterprise Architecture Process Blade
• Agile Enterprise Architecture 101
• Agile Enterprise Architecture Team Structures
• Disciplined Agile Enterprise Architecture: A Goal-Driven Approach
• Disciplined Agile Enterprise Architecture: Workflow
• DevOps Strategies: Enterprise Architecture

This blog posting has been replaced by Disciplined Agile Enterprise Architecture

Is your organization concerned with the cost and time required to adopt agile strategies?  Are you just starting out with agile and hoping to improve your chances of success by learning from the experiences of those who have gone before you?  Are you part way into your agile transformation but struggling to figure out how it all fits together?  If you answered yes to any of these questions please read on.

In this blog posting we discuss what it means to “right size” your software process to meet the needs of the unique situation that your team finds itself in.  We discuss two common anti-patterns: starting with a process framework that is much too large for your needs and starting with one that is far too small for your needs.  We argue that it’s much better to avoid these extremes and instead take a middle-ground approach by starting with a toolkit that is much closer to your actual needs.

Extreme #1: Large process repository

The first process right-sizing anti-pattern is to start with a large process repository, the classic example being IBM’s Rational Unified Process (RUP).  Although RUP is much maligned within the agile community the fact is that if you were to examine it with an open mind that there are many very good ideas promoted by RUP.  Be that as it may. The basic strategy with RUP is that you need to tailor down the process, often dramatically, to meet the unique needs of the situation your team finds itself in.  To do this requires significant process expertise, time, and money.

In practice, however, many organizations ran aground with RUP when they tailored it to be something similar to the waterfall-style processes from yesteryear that they were familiar with.  This is often referred to as RUPifall.  Another common mistake was to say to themselves “wow, there’s a lot of great ideas here, we need to do them all” and as a result they would create a process that was far too heavy to meet their needs.  In either case the problem was that they often didn’t have the process expertise required to right-size their process.  We also see this with organizations starting from frameworks such as ITIL, COBIT, and CMMI so this clearly isn’t just a RUP problem.

Extreme #2: Small methodology

At the other extreme is the right-sizing anti-pattern of starting with a small methodology, the classic example in this case being Scrum.  Although there is significant support for Scrum within the agile community, or at least among people who are new to agile, we’ve been seeing for a long time now that organizations are also running into trouble with this approach too.  With Scrum the idea is that you add in the techniques that Scrum doesn’t address to right-size your process.  Because Scrum proves to be only a very small part of the overall picture, to do this requires significant process expertise, time, and money.

In practice organizations run aground with Scrum because they don’t have the process expertise to expand it to meet their needs.  One only has to count the number of “How does X fit into Scrum?” conversations occurring in agile discussion forums online, or at user group meetings or conferences, to see that this is true.  Step back for a moment and ask yourself how much time and effort has your organization invested in trying to adopt Scrum.  Could it not have been more streamlined?

A few years ago Forrester Research discovered that the majority of organizations “doing Scrum” had actually tailored it into what they called Water-Scrum-Fall (others call this Scrumifall).  As we describe in Going Beyond Scrum this occurs for several reasons.  First, Scrum doesn’t address the full delivery lifecycle, instead choosing to focus on the construction portion of it.  As a result organizations tend to stick with what they know, a heavy project initiation phase and a heavy solution deployment phase.  Second, Scrum only addresses only a small portion of what you need and explicitly leaves technical issues up to you.  These issues include topics such testing, programming, architecture, governance, documentation, deployment and many others.  As a result teams are left to piece together a process strategy that works for them, at the very same time that they’re struggling to understand the fundamentals of agile and lean software development.  They rarely have the process expertise to do that and as a result end up having to hire hordes of expensive Scrum coaches, few of whom seem to understand the enterprise-class realities that your teams actually face.  This is a risky and expensive proposition indeed.

The Effective Middle Ground

Both of these anti-patterns represent extremes: start with something large and cut it down to size, or start with something small and build it up to meet your needs.  Why not start with something much closer to what you actually need?  Doesn’t that make a lot more sense?  Why do you need to do all this process work?  Because someone wants to sell you tooling?  Because someone else wants to sell you expensive coaching and questionable certification strategies?  Isn’t it time to consider a more pragmatic strategy?

The Disciplined Agile (DA) toolkit is a more effective middle ground.  It addresses the full delivery lifecycle, as does RUP (but not Scrum), and even gives you several choices from which to select.  Sometimes a Scrum-based lifecycle is appropriate, sometimes a Lean lifecycle is, other times a continuous delivery lifecycle is best, and sometimes an exploratory “lean start up” lifecycle is.  Different teams, different situations.  DAD starts with a lightweight approach, as does Scrum but not RUP, helping you to avoid the bloatware of RUP and filling in the numerous blanks left by Scrum.  DAD also gives you lightweight tailoring advice in the form of process goal diagrams, in many ways a cross between mind-maps and decision trees, that make your process choices explicit.  The RUP process tailoring advice, if you bother to read it at all, is rather heavy handed and the Scrum tailoring advice boils down to “you’re smart, you can figure it out and if your run into trouble hire a coach.”  Isn’t it time to abandon the extremes?

This middle ground strategy isn’t without its faults.  A challenge with DAD is that it explicitly reveals that agile software development, or as we prefer to say agile solution delivery, is complex.  This is particularly true in enterprise-class situations where teams are often facing combinations of scaling factors such as larger team size, geographic distribution, and regulatory constraints.  DAD makes it explicit that teams need to invest a bit of time up front to perform initial scoping, initial architectural modeling, and initial planning (all in a lightweight manner of course).  This sort of pragmatic thinking can be inconvenient for less-experienced developers who just want to jump in and start coding.  Because DAD promotes the philosophy of enterprise awareness it purposely bakes in strategies for governance, DevOps, and working with IT-level groups such as your enterprise architects and data management team to name a few.  This can also prove to be inconvenient for developers who want to narrowly focus on doing what’s convenient for their team as opposed to what’s best for their organization.

In Summary

The following infographic summarizes the main points in this blog posting.

We hope that you’ve found this blog posting enlightening.  Even if you are well along the way of your Scrum adoption, or of evolving your RUP-based approach, you can still benefit from switching over to DAD.  Scrum teams will find that it addresses many of the issues that you’re still struggling with, and RUP teams will find that it shows how to work in a far more lightweight manner.  Organizations will find that it provides a much better foundation from which to scale agile strategies.

In this continuing series about how the Disciplined Agile (DA) tool kit supports and enhances DevOps, we overview several strategies to help your organization adopt a Disciplined DevOps strategy:

1. Invest in your people.  In our experience 80 to 90% of your overall effort will be in helping people to learn new skills and ways of thinking and to rethink if not abandon many of the “best practices” of yesteryear.  This requires training, education, and coaching over a long period of time – most people will require many months, and sometimes years, to make the transition to this new mindset.
2. Create a safe learning environment.  Teams must be free to experiment, to try new strategies to discover what works for them in the situation that they face.  Very often this will work out well, with a few stumbles along the way, but every so often the experiment will show that the strategy just isn’t right for this team.  That should still be considered a successful experiment, learning what doesn’t work is just as valuable as learning what does, and the team should not be worried about recrimination for “failed” experiments.
3. Look at the “whole system”.  Disciplined DevOps is about more than just continuous delivery (although that is a great start) and in most enterprises it’s about more than just streamlining development and operations.  With a Disciplined DevOps mindset we strive to improve flow through the entire ecosystem, including development, operations, support, enterprise architecture, data management, release management, and most importantly to the business itself.
4. Improve locally, transform globally.  Each team, including your solution delivery teams, your enterprise architecture team, your operations staff, and many others must strive to improve and streamline the way that they work.  These local improvement efforts must be supported by a “global” transformation effort that focuses on improving DevOps across your entire ecosystem.  Every team will affect other teams, motivating them to make improvements which in turn affects how they work with others.  Your IT department is a complex adaptive system where people and teams learn and improve over time in a dynamic, evolutionary manner.  If you just focus on local improvements your DevOps effort is likely to devolve into chaos.  If you just focus on a company-wide, global transformation it is likely to get bogged down in bureaucracy.  An “improve locally, transform globally” approach is a viable middle ground that benefits from these two extremes while avoiding the disadvantages.
5. Have a communication plan (and work it).  Adopting a Disciplined DevOps strategy within your organization typically requires a lot of (often small) changes.  Although it may be clear to you why this is important it isn’t always clear to everyone else.  People need to understand why you’re making these changes, what’s in it for them, what the overall change strategy is, how far along the plan you currently are, what changes are coming soon, and so on.  You communication plan may include regular newsletters, posters overviewing key concepts, brown bag lunches where people share their experiences, electronic discussion forums, management presentations, and many more.  The keys to success are to have a constant drum beat of information, to be as open and honest about what is actually occurring, to provide opportunities to everyone to learn, and to motivate everyone to share their learnings.
6. Think long-term.  Disciplined DevOps is a journey, not a destination.  It takes time for people to adopt a new mindset, months and often years before it is truly ingrained in their way of thinking.  This paradigm shift does not occur by management fiat, nor does it occur as the result of a day or two of training (although training is important), nor does it occur because you’ve invested in new tooling.  Organizations that successfully make this paradigm shift do so by investing in their people, their process, and their tooling over the long term.