Disciplined Agile

In the Disciplined Agile (DA) toolkit data management is a Run (operational) activity that focuses on the execution of data-oriented architectures, policies, and processes. Note that the long-term planning efforts around data-oriented aspects of your organization are part of your Enterprise Architecture efforts. Similarly, development of the data-oriented aspects of your organizational eco-system is addressed by solution delivery teams.

Because data management is an important aspect of your Run endeavors it will be affected by your Disciplined DevOps strategy. Our experience is that in addition to the general DevOps strategies describe previously, there are several data management strategies that support DevOps:

• Data and information guidelines. A straightforward way to promote greater consistency in the development and application of data and information sources is to have common guidance that teams will adopt and then follow. This guidance, including both standard policies and guidelines, will need to be defined, supported, and evolved over time in a collaborative and open manner.
• Quality data sources. Your production data sources, including files, databases, and data feeds, should be high quality assets that are easy to work with. When it comes to data sources of record it is particularly important for them to be of high quality so that they are easy to work with and evolve. Unfortunately this is often little more than fanciful thinking in many organizations. With a Disciplined DevOps mindset teams realize that they should be very careful about increasing the technical debt within their data sources, and more importantly invest in the effort to pay down any technical debt that they find.
• IT intelligence.  IT intelligence is the creation, support, evolution, and operation of  data warehouse (DW)/business intelligence (BI) solutions that support the management and governance of your IT efforts. From a Disciplined DevOps perspective this there are two important aspects of IT intelligence: development intelligence that provides insight into how delivery teams are working and operational intelligence that provides insight into what occurs in production. The automated team dashboards provided by many development platforms are a simple form of development intelligence, a more sophisticated (and useful) strategy is to combine information from various development tools to display it in an integrated dashboard for the team, and more sophisticated yet is to roll up/combine information from different delivery teams into a portfolio management dashboard. Similarly your operations team may have individual dashboards for each solution, they may combine information being generated by individual solutions into an integrated dashboard, and better yet share that information for an IT portfolio management view.

In the next blog posting in this series we will explore DevOps strategies pertaining to enterprise architecture.

In addition to the general release management strategies described previously, the general DevOps strategies, and the construction-focused DevOps strategies (including continuous deployment) there are several other release management strategies that support DevOps:

• Integrated deployment planning.  From the point of view of development teams, deployment planning has always required interaction with an organization’s operations and release management staff; in some cases, via liaison specialists within operations often called release engineers. When you adopt a Disciplined DevOps mindset, you quickly realize the need to take a cross-team approach to deployment planning due to the need for operations staff to work with all of your development teams. This isn’t news to operations staff, but it can be a surprise to development teams accustomed to working in their own, siloed environments (luckily this strategy is built into DAD’s Move Closer to a Deployable Release process goal). If your team is not doing this already, you will need to start vying for release slots in the overall organizational deployment schedule.
• Standard development and testing environments based on production. Development teams know that the greater the consistency between their development, testing, and production environments the easier it is to test and deploy. In multi-team environments the implication is that this will result in defacto standardization of many aspects of your environments. Developers may choose different development tools, but aspects of the infrastructure such as operating systems, application servers, middleware, databases and so on will become consistent over time to streamline the overall release process.
• Release service streams. A key tenant of DAD is that every team is unique, and an implication of that is that some teams will need more help than others. Teams will produce different levels of quality, they will have different amounts of automation, they will have different release cadences, and so on. As a result your release management strategy needs to be flexible enough to address these different situations. One way to do so is to offer different server streams, or service levels as it were, to solution delivery teams. For example, you may have a basic release management service stream where release management engineers actively help delivery teams to deploy their solutions into production and even help them to start automating some of their processes. At the other end of the spectrum you may have a continuous delivery service stream for delivery teams that have fully automated their testing and deployment processes and that can be trusted to successfully deploy on their own. And of course you could have several other service streams between those two extremes. The advantage of this approach is that it is very flexible albeit at the cost of slightly greater scheduling complexity.
• Release blackout periods. Some organizations have periods of time where they choose not to release new functionality into production unless it is absolutely critical. These blackout periods typically occur during high-volumes of business transactions. For example, many North American retail companies will have blackout periods between mid-November and early January for the holiday sales seasons. Many organizations will have blackout periods near the end of their fiscal years to enable them to focus on the process required to close out the year.
• Shared practices. Although this is really a process improvement issue, it’s worthwhile to point out that whoever is involved with release management should actively strive to share effective practices between teams. Sharing learnings across teams is an important aspect of enterprise awareness.

In the next blog posting in this series we will explore data management strategies that support a DevOps mindset.

In this blog posting we describe four general release management strategies that support DevOps. These strategies, from least effective to most effective, are:

1. Release windows (slow cadence). A release window is a period of time during which one or more teams may release into production. A release slot is subset within that release window (and may be the entire window) during which a team may deploy their solution into production. For example, your organization may have a policy that production releases occur between 1am and 5am on Saturday evenings (the release window) and that up to four releases may occur during that window (the release slots). In lean terms, a release slot is effectively a Kanban card that allows a team to deploy. Release windows tend to align with periods where system usage is lower, although in the modern world of global 24/7 operations these periods have all but disappeared. With a slow cadence approach to this strategy the release windows occur far apart, as seldom as once a week or even longer. The advantages of this approach are that it provides a consistent release cadence to business stakeholders and it provides consistent release date targets for delivery teams. The primary disadvantage with slow cadence release windows is that they become bottlenecks for release management processes that need to support multiple teams. There are only so many release slots available during each window and this number can be easily exceeded, forcing teams to aim for a future release window. This problem becomes exacerbated when teams start to move to a continuous delivery strategy.
2. Release train. The idea with a release train is that every team involved with that “train” has the same release cadence – for example this train releases once a quarter, or once a month, or even once a week. This strategy is commonly used in large programs, or teams of teams, where the individual teams are each working on part of a larger whole. Having the common drumbeat of a release train provides a consistent release schedule for stakeholders and serves as a rallying point for development teams. The train metaphor works quite well in practice. If your team misses the release date, if you miss the train, then the train goes on without you and you need to wait for space on the next on. Dependencies are also respected. For example, if several components need to ship together they must all go on the same train (similar to a family taking a trip together). The primary disadvantage is that development teams are constrained to a common release schedule, making it difficult to support lean or continuous delivery strategies. A potential disadvantage is that release trains may also suffer from the bottleneck problems of slow cadence release windows.
3. Release windows (quick cadence).  To support continuous deployment, particularly across many delivery teams, you will need a large number of release slots. The implication is that you will also likely need more release windows more often. The advantage of quick cadence release windows is that they are less likely to suffer from the bottleneck challenges associated with slow cadence release windows and release trains.
4. Continuous release availability. With this approach delivery teams are allowed to release their solutions into production whenever they need to. In many ways this is simply an extension of the release window strategy to be 24/7. This is the only strategy that truly supports continuous delivery. To make it work a host of DevOps practices are required, such as fully automated deployment, fully automated regression testing, feature toggles, self-recovering components, and many others are required.

Our experience is that most enterprises today employ a slow cadence release window approach although are starting to evolve into the quick cadence version of this strategy. This is usually motivated by the adoption of agile techniques by solution delivery teams and more often than not by continuous delivery practices. We also see large programs take a release train approach, a strategy pioneered in the 1990s by large software companies such as Microsoft and Rational that sold software suites comprised of many products that needed to be shipped together. In recent years the OpenUP and SAFe frameworks have popularized the release train strategy. The strategy of continuous release availability is commonly used in advanced DevOps organizations such as Etsy and Amazon.

An important point to be made is that there are several options available to you, each of which has its advantages and disadvantages.  No single approach is perfect, and no single approach works in all situations.  You not only need to have choices, it’s good to have choices.

The next blog posting in this series continues with more release management DevOps strategies.

In this blog posting we describe two issues for organizing your release management strategy: How to scope release management and how to organize the team.

There are two fundamental issues to consider when scoping your release management efforts:

1. Paradigm support. Will your release management process focus on supporting one paradigm, such as agile/lean teams or will it provide a more holistic strategy to also support agile/lean teams, traditional teams, iterative teams, and even ad-hoc teams? Many people who are currently writing about release management tend to focus on a single paradigm, although they may not explicitly state this, but the reality is that most enterprise-class organizations need multi-paradigm support in practice.
2. Domain support. Will your release management process focus on IT-related issues or will it address the full range of business-related release issues? IT-related release issues include deploying new software and hardware into production. Business release issues may include marketing campaigns, training your sales force, and organizing external support mechanisms for end users to name a few. This is particularly important for commercial solutions being produced for the end customer of your organization.

These two issues lead us to the following quadrant chart depicting the potential scope for release management:

From a Disciplined DevOps point of view we of course promote a Holistic Enterprise scoping strategy. Whatever scoping strategy you choose your release management strategy will need to be able to support the scaling situations faced by your delivery teams. This includes teams of various sizes, different levels of geographic distribution, dealing with different levels of domain and technical complexity, teams that are organizationally distributed, and teams in compliance situations.

There are three strategies to consider for organizing your release management team:

1. Operations led. In many small to medium-sized organizations release management is one of many activities that are performed by the operations team. With this approach a “release team”, in some cases an individual, is put together to release a solution on a project-by-project basis. Although there is often a hand-off point from the development team to the operations team, the operations team may require several members of the development team to be actively involved with the deployment. The advantages of having operations manage releases are that they are very familiar with the current state of your production environment and they know what other releases are happening in parallel (if any) and thereby have an integrated view of the overall situation. The primary disadvantage is that they will not know the intricacies of the new release of the solution, hence the need to include development team members.
2. Separate release team. Larger organizations will often have an explicit release management team, often a subgroup of their operations department. The advantages of a separate team include the ability to grow expertise in release management, familiarity with your production environment, and the ability to manage releases in an integrated manner. The disadvantages are the lack of familiarity with solution(s) being released and the potential to inject overhead into the overall release process.
3. Delivery team led. This approach is common in very small organizations that do not have separate operations teams and in organizations delivery teams have adopted at very disciplined approach to DevOps that supports the practice of continuous deployment. The advantages of a delivery team approach are that that team is very familiar with how the solution is built and they are given greater flexibility to deploy as needed into production. The disadvantages are that there is a risk of deployment collisions in multi-team environments and integration problems in production between disparate systems. Luckily these disadvantages can be addressed via a combination of development-oriented DevOps practices and the following release management practices.

There is no such thing as a “best practice”, except perhaps from a marketing point of view.  All practices (and strategies) are contextual in nature.  In some situations a practice works incredibly well and it other situations a practice can be the kiss-of-death.  Two of the philosophies behind the Disciplined Agile (DA) tool kit are that choice is good and that you should understand the trade-offs of the options available to you.  In short, practices are contextual in nature and you should strive to adopt the ones that work best for the situation that you find yourself in.

Let’s work through an example.  A hot button for many people are strategies around cost estimation, typically used for budgeting and forecasting purposes.  In DAD initial estimation is addressed by the Plan the Release process goal, the goal diagram for which is shown below.  As you can see with the Estimating Strategy factor there are several options available to you.  We’re not saying that these are all of the potential estimating options available, but we are saying that this is a fairly good representative range.  The arrow on the left-hand side of the strategies list indicates that the strategies towards the top of the list are generally more effective for initial planning than the strategies towards the bottom of the list.  Your mileage may vary of course.

The following table summarizes the trade-offs involved with two of the estimating strategies, in the case formal point counting (such as function points) and an educated guess by an experienced individual.  One of the reasons why Choose Your WoW! is so thick is that much of it is tables like this communicating the trade-offs of the hundreds of practices and strategies encapsulated by the toolkit.  As you can see, there are advantages and disadvantages to both strategies.  You can also see that there are situations where each strategy potentially makes sense.  Although you may not like a given strategy, I personally abhor formal point counting, you should still respect the fact that the strategy is viable in certain situations (perhaps not yours).

There are several reasons why DAD’s goal-driven approach is important:

1. It makes your choices explicit.  If your team wants to truly own your process then it first needs to know what choices are available to it.
2. It makes it clear that practices are contextual in nature.  No practice is perfect for all situations, every single one has advantages and disadvantages.  Are you choosing the ones that are most appropriate for your situation?
3. You can have more coherent discussions of the trade-offs that you’re making.  We have endless religious battles in the IT industry around process-related topics.  This often happens because people choose to focus on the benefits of their favorite practices and to downplay the disadvantages (or worse yet are oblivious to them).  To help your team move to more effective practices it’s important to recognize the trade-offs involved with each, to then discuss them rationally, and then decide to adopt the strategy that is most viable given your situation.  Note that this doesn’t necessarily mean that you’re going to adopt the best practice from the start, but that instead you’ll adopt the best one that you can right now.  Later on, perhaps as the result of a retrospective, you’ll decide to make improvements to your approach (in the case of process factors where the strategies are ranked by effectiveness, your team may choose to adopt a strategy higher in the list).
4. Improves the effectiveness of retrospectives.  During a retrospective it is fairly straightforward to identify potential problems that you’re facing, what isn’t as easy is identifying potential solutions.  You can improve retrospectives by having these process goal diagrams available to you to suggest potential strategies that you should considering adopting.
5. You can avoid reinventing existing practices.  Many very smart and very experienced people have found ways to deal with the same challenges that you’re facing today.  Furthermore, many of them have shared these strategies publicly.  If you don’t know that these strategies exist you are at risk of wasting time reinventing them, time that could be better spent adding real value to your organization.
6. It enables scaling.  Teams in different situation will make different process decisions.  Although teams at scale, perhaps they are large teams or geographically distributed, will follow many of the same practices as small co-located teams they will also adopt a few strategies that make sense for them given their situation.   DAD’s process goals provide the insight that teams need to understand how they can address the challenges associated with the scaling factors that they face.

For a more detailed discussion about the challenges around “best practices”, you may find the article Questioning Best Practices to be an interesting read.