Disciplined Agile

On August 8th I facilitated a workshop at Agile 2018 that focused on agile architecture. I promised everyone, we had over 150 people in the workshop, that I would post the pictures of their strategy canvases online so that they could have copies of it.  So here’s my promised blog!

How architecture fits into agile ways of working (WoW) has been an important topic from the very beginning.  As with all other aspects of agile, the Disciplined Agile (DA) toolkit provides you with choices – because every team is unique and faces a unique situation, they need to choose their WoW to reflect this (we like to say that context counts).  To enable teams to effectively choose their WoW they need to know what choices they have available to them (we also believe that choice is good) and what the trade-offs of those choices are (because there’s no such thing as a best practice).

So in this workshop we worked through twelve agile architecture strategies.  Granted, there’s a lot more than this, but we only had 75 minutes so I lead everyone through some of the key strategies.  This blog overviews each strategy and then shares the strategy canvases that the class developed.  Our approach was to assign each strategy to several tables and then ask each group to discuss the advantages of the strategy, the disadvantages, and when you would use it. There are pictures below of each strategy canvas – it’s important to note that I don’t agree with all of the sticky notes, but for the most part it’s solid stuff.  Our upcoming book, trending to arrive in October 2018, explores these strategies (and more).

During the workshop we explored three fundamental questions:

1. How Do You Explore Your Initial Architecture Strategy?
2. How Do You Bring Architecture Skills Into the Team?
3. How Do You Evolve Your Architecture?

How Do You Explore Your Initial Architecture Strategy?

Each group was given one of three initial architecture strategies to discuss:

1. No modeling
2. Lightweight modeling
3. Heavy modeling

No modeling

At the beginning of our project/endeavor we will perform no architecture modeling or exploration at all.  We believe that all aspects of the architecture will emerge over time during Construction.

Lightweight modeling

At the beginning of our project/endeavor we will invest a bit of time, potentially several hours or even a few days, to perform some lightweight agile architecture modeling.  We believe that we should think through the high-level architectural issues now but that the design details will emerge over time. Our “models” will be created using inclusive modeling tools such as whiteboards, paper or sticky notes. We’re likely to create one or more high-level sketches, each of which explores an architectural view (such as the technical architecture, the data architecture, the communications architecture, and so on).

Heavy modeling

At the beginning of our project/endeavor we will invest a fair bit of time, potentially several weeks or even a few months, to work through the architecture of our solution.  We believe that we need to think through most, if not all, aspects of the architecture in detail before we can safely proceed with Construction.  Our models may be either visual (i.e. UML component diagrams, Free-form architecture diagrams, Data Models, …) or textual (e.g. white papers) in nature.  Our visual models will be supported by sufficient documentation to capture the details overviewed by the diagrams.  Our models will be captured with software-based diagramming tools (e.g. Visio, OmniGraffle), documentation tools (e.g. Word processors or Wikis), or modeling tools (e.g. Enterprise Architect).

How Do You Bring Architecture Skills Into the Team?

Each group was given one of five architecture skills/roles strategies to explore:

1. No architect
2. Part-time architect
3. Modeler architect
4. Developer architect
5. Architecture owner

No architect

On our project/endeavor we do not have anyone on the team responsible for architecture work nor do we have anyone with sufficient architecture skills to do so.  Luckily we are a group of smart, experienced developers and we believe that we will be able to evolve the architecture of our solution through teamwork and experimentation.

Part-time architect

On our project/endeavor we have someone on our team acting as a part-time architect.  This person is very experienced and is a member of our corporate architecture team, supporting several solution delivery teams including our own.  The part-time architect guides our team in architecture decisions, providing advice, educating us in any existing corporate architecture guidance, and reviewing any work that we do on our own.

Modeler architect

On our project/endeavor we have someone with deep architecture experience and skills.  In the past they were a developer but over time they worked their way out of that role to become an architect.  Their job is to work through the architectural strategy for the team, describe that architecture strategy to us, and develop and maintain sufficient architecture models and documentation throughout the lifecycle.  Although they are technically adept and well versed in the domain, they do not have technical experience with some of the platforms and technologies that the team is likely to work with. 

Developer architect

On our project/endeavor we have someone who is a full-time member of our team who has deep architecture experience as well as development skills. They are responsible for formulating the architecture strategy for the team although will often work with other team members to get their input into architecture decisions. When they are not doing architecture work they are an active developer on the team.

Architecture owner

On our project/endeavor we have someone in the role of Architecture Owner (AO).  This is a full-time team member who has architecture experience, is well versed in our corporate architecture guidance/strategies, and also has development skills.  The AO:

• Guides/facilitates the team through architectural strategy and related work
• Mentors/coaches team members in architecture thinking and skills
• Work with our corporate architecture team as needed to ensure that our team is following, to the extent appropriate, the overall architectural vision
• Is an active developer on the team when they are not performing their other responsibilities

How Do You Evolve Your Architecture?

Each group was given one of four architecture evolution strategies to explore:

1. Architectural Guidance
2. Architecture Spikes
3. Walking Skeletons
4. Just-in-time (JIT) Model Storming

Architectural Guidance

One of the approaches that our team uses to evolve our architecture is refer to our organization’s existing architectural guidance. We typically do this by talking with someone on our team familiar with that guidance, such as an Architecture Owner (AO).  This person may decide to look up the current version of the architecture guidance as needed (hopefully this is captured in an agile, lightweight manner).  Such architecture guidance may include:

• Technologies we’re moving towards
• Technologies we’re moving away from
• Technologies that we mustn’t use
• Previously identified architecture experiments to run (important in multi-team environments to coordinate learning efforts)
• Results from architecture spikes performed by other teams

Architecture Spikes

One of the approaches that our team uses to evolve our architecture is to run architecture spikes (or simply called spikes). Whenever we run into an architectural issue, such as how a technology works within our environment or how a combination of technologies work together, we write a bit of code to explore the issue.  This is typically throwaway prototyping code because we want to quickly experiment to gather sufficient data to make a better-informed decision.  Spikes typically take a few hours to a few days. A spike can be thought of as a very small proof of concept (PoC).

Walking Skeletons

One of the approaches that our team uses to evolve our architecture is to develop a walking skeleton, also known as a working skeleton or steel frame, of our solution early in Construction.  We do this by focusing on a small collection of requirements early in the lifecycle that implement sufficient functionality to address our highest-risk items so as to show that our architectural strategy works in practice.  Once the walking skeleton is in place we spend the rest of Construction putting the functional flesh onto it.

Just-in-time (JIT) Model Storming

One of the approaches that our team uses to evolve our architecture is to explore requirement and design details at the last most responsible moment.  We do this via light-weight agile modeling (sketching, sticky notes, …).  Model storming sessions are often impromptu and last for several minutes, often starting with a request along the lines of “Hey, can you help me work through this…”.

Parting Thoughts

After the workshop I received a lot of great feedback from attendees.  It really resonated with people and seemed to provide them with a lot of value.  I was glad to hear that.  

Further Reading

• Architecture Owner
• Identify Initial Architecture Strategy
• Enterprise Architecture
• Agile Architecture Strategies

For a long time now we’ve been applying what’s often called rolling wave planning with our clients. Rolling wave planning is applied in several areas of the Disciplined Agile (DA) toolkit, including release planning by a delivery team, technology roadmapping, and product roadmapping to name a few. This article explores how to apply rolling wave planning in a pragmatic manner to technology roadmaps.

An important aspect of your enterprise architecture efforts is to provide architectural guidance, both business and technical guidance, to your organization. One of the key artifacts that enterprise architects will create is a technology roadmap that, as the name suggests, provides guidance as to the proper application of technologies within your organization. This roadmap will often supplement any “as is” and “to be” models that the enterprise architects create.

Technology roadmaps are often evolved following a rolling wave planning approach. Figure 1 depicts an example of a technology roadmap, the goal of which is to  give technical direction to solution delivery teams so as to provide safety rails around technical experimentation.

Figure 1. A technology roadmap in September 2016.

Notice how this roadmap addresses several categories of technical issues:

1. Planned upgrades. Development teams need to know what aspects of your infrastructure will be evolving and when. Upgrades that are soon to occur have been assigned dates as these changes are very likely going to impact several development teams.
2. Experiments. From an agile point of view the most interesting thing is the list of experiments that the enterprise architects are overseeing. These experiments usually occur within one or more of the development teams underway in this company. In this case the EAs are coordinating and guiding the experiments, ensuring that the teams focus on experiments that reflect the overall direction of the organization. In some cases you may choose to have an explicit proof of concept (PoC) project to validate a new technology, a common approach for major infrastructure components or expensive packages. Either way, by planning for explicit experimentation you bring greater discipline to your learning and architectural evolution efforts.
3. Reusable infrastructure. This company has a reuse engineering effort underway where common architectural components, in this case microservices, are built and then made available to development teams. Once again, the closer to being released the reusable components are the more likely it is to have a target date for their release.
4. Retirements. An aspect of technology roadmaps that are often forgotten are the plan to retire existing systems and infrastructure components. An important part of paying down technical debt within organizations is the consolidation of your IT infrastructure – reducing the number of technologies that you’re using, removing redundant systems, and so on. Such retirement efforts can take months and even years. We see in Figure 1 that SQLServer is currently being retired from service, development teams are migrating to approved database technologies, and that it is slated to be completed by February 5^th. In this case the enterprise license for SQLServer ends in April 2017 so they’re hoping to be completely off of it two months before that.

What Should the Planning Horizon Be?

Technology roadmapping tends to have between a six month and three year planning horizon depending on what is being planned for. For example, experiments are typically planned out a few months in advance as they are often driven by the needs of development teams. Major upgrades are typically planned on a horizon of six months to a year as this reflects the rate of change of many technologies. Retirements might typically planned for years in advance, particularly when the retirement could impact multiple systems.

How to Capture Technology Roadmaps

Technology roadmaps are typically captured in text format as you see in Figure 1 above, although a timeline format (as we say with product roadmaps) are often used for executive presentations.

This sort of planning is only one of several things that your enterprise architecture team will do of course. In addition to actively guiding development teams and working with senior stakeholders, the enterprise architects are also maintaining current and to-be models and are hopefully producing code examples for how to work with new architectural components.

On May 17 2016 I gave a webinar entitled Agile Enterprise Architecture: Disciplined and Pragmatic Strategies (at this link you can watch a recording and download a PDF of the slides). During the webinar I received many questions, some of which we had time to answer but some which we did not.  Regardless, here are all of the questions and my answers.  I’ve organized them into the following topics:

1. Long vs. Short Term
2. Deferring Commitment
3. Transitioning to Agile EA
4. General

1. Long vs. Short Term

What about longer term? How do you see the role of EA on portfolio creation?  As you can see in Disciplined Agile’s Enterprise Architecture process blade one of the primary tasks of the EA activity is to consider the longer term. The enterprise architects will collaborate to do so, and will help bring their vision to their stakeholders in various ways.  As you can see in the workflow, the EAs will interact with the Portfolio Management effort (and many others).

How do you balance the needs of today vs. the needs of tomorrow? Would you wait for business to identify it as a priority, which may be too late… or look to get ahead of the curve and build something that can scale for anticipated growth? Experienced enterprise architects will definitely think about the long term needs of the organization. Disciplined Agile EAs will do so but will wait to act to the most appropriate time to do so.

2. Deferring Commitment

Can you point us to some places to read more about deferred commit? Deferring commitment to the last most responsible moment is a concept that comes from lean software development. I recommend the writings of Mary and Tom Poppendieck as well as the book SDLC 3 by Mark Kennaley.

If we defer commitment, how about if we need infrastructure, procurement and so on to implement that architecture? This deferring will delay the entire implementation? No. The advice is to defer commitment to the last most responsible moment.  You need to understand the needs of your stakeholders, which in Disciplined Agile include a wide range of people, and then act accordingly. The problem is that traditionalists will act too early and thus add unnecessary risk.

A good observation from Valentin Tudor-Mocanu is that to be able to defer decisions we need a design that separates the concerns (adaptive design).

3. Transitioning to Agile EA

Is there a strategy to change the mindset of enterprise architects on a large organization?  Yes. It first begins with education.  We offer a one-day workshop called Agile Enterprise Architecture: Disciplined and Pragmatic Strategies that does a very good job of working through the fundamentals.  The next step is coaching of the EA team themselves, and of the teams that they interact with, to help them gain the mindset and habits required to work in a more effective agile/lean manner.  We also provide such coaching services.

One specific question related to a service or consulting organization: One of the specific requirements for us to is to deliver architecture as a deliverable. How can agile can help us?  Can we apply sprints there?  Can we apply any architecture principles? I’ll assume you meant an architecture model as a deliverable. The answer is yes, you can easily create an architecture model in an agile manner. I highly suggest that you visit the Agile Modeling site and read the advice about agile architecture strategies and to agile documentation strategies.

4. General

Can you talk about metrics in agile EA and Architecture in general?  In the Disciplined Agile (DA) toolkit we suggest that you adopt a lightweight Goal-Question-Metric (GQM) approach to your metrics strategy. The fundamental idea behind GQM is that you first identify a goal that you would like to achieve, a set of questions whose answers are pertinent to determining how well you’re achieving that goal, and then the metric(s) that could help you to answer each question. In short, the metrics that you gather will vary based on your need.

Given that Feedback cycle plays an important role, why does DAD say “Reduce feedback cycle”? Is there a pre-defined cycle frequency?  You should find the blog posting Reducing the feedback cycle requires discipline to be an interesting read.

Some forms of architecture (UX/UI) need a different cadence than the delivery team follows. What is your recommendation for handling this… UX research, etc? I disagree with your premise. I’ve heard this claim in the past and it’s never held water with me.  About 10 years ago I wrote an article entitled Introduction to Agile Usability that overviews how UX activities can appear in the agile lifecycle.  The Lean UX community has clearly gone further than what I originally wrote so you might want to go poking around on the web a bit.

When the business case says “many years before payback”, doesn’t that usually mean that the business case is missing some important elements? (or that the org is really small) ?  My experience is that the business case is usually crap in these situations. There in fact a few things in the software world that may have a multi-year payback period, operating systems come to mind, but they are more the exception to the rule.  The problem with multi-year payback schemes in the software world is that the ecosystem changes so quickly.  During the several years it takes for payback your needs change, there are new options on the market, your organization structure and direction may change, and so on.

I realize I’m asking a history question here, but regarding deferring commitment, a superb method for this that I have often used in the past comes from RUP: architecture mechanisms. (I should know this but) Is this in DAD?  If you read the Enterprise Architecture process blade article you will see that it supports several strategies for capturing and communicating EA ideas such as reference architectures, models, architectural runways, interfaces, and many others. You might also find the Reuse Engineering process blade to be of interest.

One of the age-old debates in the software world is whether software architects need to write code.  We suspect that as an industry we’ll never reach consensus on this topic. Here are our thoughts on the subject.

Short Answer:

Hell yes!

Detailed Answer:

In the following table we list the advantages, disadvantages, and considerations (when does the strategy makes sense) to compare whether a software architect should write code or not.  You may recognize this approach from our book Disciplined Agile Delivery.

In the Disciplined Agile (DA) toolkit we’ve made it very clear that we expect Architecture Owners to be actively involved with the development of the solution.  On Disciplined Agile teams the Architecture Owner is effectively a team member with additional responsibilities around leading the team through architecture decisions, in coaching them on architecture skills, and in working closely with your Enterprise Architecture team (if any) to ensure their development team understands and is working towards your organization’s technical roadmap.

We’re often told that it isn’t realistic to expect architects to write code.  Invariably this is coming from people who are currently working in traditional IT organizations that have very well-defined roles, IT organizations that more often than not are struggling to be effective.  Our response is always the same – Really?  Are development teams following your architectural strategy?  Are they eager to work with you, or are they forced to work with you?  This generally leads to a discussion that reveals that things aren’t going so well for these architects in practice, and sometimes leads to a positive discussion as to how we can move towards a more effective approach for them.  They kind of approach described in the Disciplined Agile (DA) toolkit.

Additional Reading:

• In the mid-1990s James Coplien captured the organizational pattern Architect Also Implements.
• The Architects Don’t Code anti-pattern on Ward Cunningham’s C2 wiki is also a valuable read.

Enterprise architecture, when performed in a disciplined agile manner, is an important enabler of agile software delivery.  This is true for several reasons:

1. Common architecture enables agile teams to focus on value creation.  A common enterprise architecture enables reuse across delivery teams.  When agile teams have high-quality assets – such as micro-services, legacy data sources, and frameworks – available to reuse they are able to focus on creating new value for their stakeholders and not on reinventing new versions of existing assets.
2. Common technical guidance enables greater consistency between teams.  When team follow effective, common conventions it results in greater quality.   This makes it easier to learn about assets that are new to them, in particular existing source code, and to evolve those assets as needed.  Greater consistency also makes it easier for people to move between teams because it will be easier for them to come up to speed on what the new team is doing and to share their skills with those team members.
3. Agile architectures enable disaggregation.  When your solutions are built from loosely coupled, highly cohesive components it is easier to spread development work across smaller teams.  This reduces overall risk and organizational complexity, which in turn reduces time-to-delivery.  It also allows agile teams to focus on what they’re building instead of coordinating with other teams or mocking out the (undelivered) work of other teams.
4. Common infrastructure enables continuous delivery.  When there is a common technical infrastructure to IT delivery teams to deploy into it is easier to deploy.  The easier it is to deploy, the more often it makes sense to deploy.
5. Enterprise architecture scales agile.  A disciplined agile approach to enterprise architecture enables organizations to scale agile strategies “horizontally” across their entire IT department.