Project Management

Disciplined Agile

by , , , , , ,
#ChooseYourWoW | #ContinuousImprovement | #Kaizen | #ProcessImprovement | Adoption | agile | Agile certification | agile transformation | Analogy | Architecture | architecture | book | Business Agility | Certification | Choose your WoW | CMMI | Coaching | Collaboration | Compliancy | Configuration management | Construction phase | Context | Continuous Improvement | COVID-19 | Culture | culture | DAD | DAD discussions | DAD roles | Data Management | database | DevOps | Discipline | disciplined agile delivery | Documentation | DW/BI | Enterprise Agile | Enterprise Architecture | Enterprise Awareness | Essence | Evolving DA | Experiment | Financial | GDD | Geographic Distribution | global development | Goal-Driven | goal-driven | goals | Governance | Guideline | Hybrid | Improvement | inception | Inception phase | Kanban | Large Teams | Lean | Lifecycle | lifecycle | Metrics | mindset | News | News and events | Non-Functional Requirements | non-functional requirements | Operations | Outsourcing | People | Philosophies | Planning | PMI | PMI and DA | Portfolio Management | Practices | Principle | Process | process improvement | Product Management | Product Owners | Program Management | Project Management | Promise | quality | Release Management | Requirements | requirements | Reuse Engineering | Risk management | RUP | Scaling | scaling | scaling agile | Scrum | Support | Surveys | Teams | Technical Debt | Terminology | Testing | testing | Toolkit | Transformation | Workflow | show all posts

About this Blog

RSS

View Posts By:

Scott Ambler
Glen Little
Mark Lines
Valentin Mocanu
Daniel Gagnon
Michael Richardson
Joshua Barnes

Recent Posts

Failure Bow: Choosing Between Life Cycles Flowchart Update

Evolving Disciplined Agile: Guidelines of the DA Mindset

Evolving Disciplined Agile: Promises of the DA Mindset

Evolving Disciplined Agile: Principles of the DA Mindset

Evolving Disciplined Agile: The DA Mindset

DevOps Strategies: Release Management Part 1

DevOps Practices - Release Management

In this blog posting we describe four general release management strategies that support DevOps. These strategies, from least effective to most effective, are:

  1. Release windows (slow cadence). A release window is a period of time during which one or more teams may release into production. A release slot is subset within that release window (and may be the entire window) during which a team may deploy their solution into production. For example, your organization may have a policy that production releases occur between 1am and 5am on Saturday evenings (the release window) and that up to four releases may occur during that window (the release slots). In lean terms, a release slot is effectively a Kanban card that allows a team to deploy. Release windows tend to align with periods where system usage is lower, although in the modern world of global 24/7 operations these periods have all but disappeared. With a slow cadence approach to this strategy the release windows occur far apart, as seldom as once a week or even longer. The advantages of this approach are that it provides a consistent release cadence to business stakeholders and it provides consistent release date targets for delivery teams. The primary disadvantage with slow cadence release windows is that they become bottlenecks for release management processes that need to support multiple teams. There are only so many release slots available during each window and this number can be easily exceeded, forcing teams to aim for a future release window. This problem becomes exacerbated when teams start to move to a continuous delivery strategy.
  2. Release train. The idea with a release train is that every team involved with that “train” has the same release cadence – for example this train releases once a quarter, or once a month, or even once a week. This strategy is commonly used in large programs, or teams of teams, where the individual teams are each working on part of a larger whole. Having the common drumbeat of a release train provides a consistent release schedule for stakeholders and serves as a rallying point for development teams. The train metaphor works quite well in practice. If your team misses the release date, if you miss the train, then the train goes on without you and you need to wait for space on the next on. Dependencies are also respected. For example, if several components need to ship together they must all go on the same train (similar to a family taking a trip together). The primary disadvantage is that development teams are constrained to a common release schedule, making it difficult to support lean or continuous delivery strategies. A potential disadvantage is that release trains may also suffer from the bottleneck problems of slow cadence release windows.
  3. Release windows (quick cadence).  To support continuous deployment, particularly across many delivery teams, you will need a large number of release slots. The implication is that you will also likely need more release windows more often. The advantage of quick cadence release windows is that they are less likely to suffer from the bottleneck challenges associated with slow cadence release windows and release trains.
  4. Continuous release availability. With this approach delivery teams are allowed to release their solutions into production whenever they need to. In many ways this is simply an extension of the release window strategy to be 24/7. This is the only strategy that truly supports continuous delivery. To make it work a host of DevOps practices are required, such as fully automated deployment, fully automated regression testing, feature toggles, self-recovering components, and many others are required.

Our experience is that most enterprises today employ a slow cadence release window approach although are starting to evolve into the quick cadence version of this strategy. This is usually motivated by the adoption of agile techniques by solution delivery teams and more often than not by continuous delivery practices. We also see large programs take a release train approach, a strategy pioneered in the 1990s by large software companies such as Microsoft and Rational that sold software suites comprised of many products that needed to be shipped together. In recent years the OpenUP and SAFe frameworks have popularized the release train strategy. The strategy of continuous release availability is commonly used in advanced DevOps organizations such as Etsy and Amazon.

An important point to be made is that there are several options available to you, each of which has its advantages and disadvantages.  No single approach is perfect, and no single approach works in all situations.  You not only need to have choices, it’s good to have choices.

The next blog posting in this series continues with more release management DevOps strategies.

 

Posted by Scott Ambler on: March 09, 2015 06:17 PM | Permalink | Comments (0)

Put Practices in Context: #NoBestPractices

There is no such thing as a “best practice”, except perhaps from a marketing point of view.  All practices (and strategies) are contextual in nature.  In some situations a practice works incredibly well and it other situations a practice can be the kiss-of-death.  Two of the philosophies behind the Disciplined Agile (DA) toolkit are that choice is good and that you should understand the trade-offs of the options available to you.  In short, practices are contextual in nature and you should strive to adopt the ones that work best for the situation that you find yourself in.

Let’s work through an example.  A hot button for many software developers are strategies around cost estimation, typically used for budgeting and forecasting purposes.  In DAD initial estimation is addressed by the Develop Initial Release Plan process goal, the goal diagram for which is shown below.  As you can see with the Estimating Strategy factor there are several options available to you.  We’re not saying that these are all of the potential estimating options available, but we are saying that this is a fairly good representative range.  The arrow on the left-hand side of the strategies list indicates that the strategies towards the top of the list are generally more effective for initial planning than the strategies towards the bottom of the list.  Your mileage may vary of course.

Initial release planning

The following table summarizes the trade-offs involved with two of the estimating strategies, in the case formal point counting (such as function points) and an educated guess by an experienced individual.  One of the reasons why the DAD book is so thick is that much of it is tables like this communicating the trade-offs of the hundreds of practices and strategies encapsulated by the toolkit.  As you can see, there are advantages and disadvantages to both strategies.  You can also see that there are situations where each strategy potentially makes sense.  Although you may not like a given strategy, I personally abhor formal point counting, you should still respect the fact that the strategy is viable in certain situations (perhaps not yours).

 

Strategy Advantages Disadvantages Considerations
Formal point counting
  • Fulfills contractual obligations in some situations – for example, the US Government often requires function-point based estimates.
  • Provides a consistent way to compare projects and team productivity.
  • Often increases the political acceptability of the estimate due to the complexity of the effort to create it.
  • Greatly extends the Inception phase effort.
  • Provides a scientific façade over the estimation activity, even though estimation is often more of an art in practice.
  • Reduces team acceptance of the estimate because the estimate is typically produced by a professional estimator.
  • Historical data won’t exist for new technology platforms or development techniques, requiring you to guess the value of some complexity factors.
  • Provides a mechanism to compare the productivity of development teams, which can motivate them to over estimate and thereby decrease comparability and the value of your historical database.
  • Total cost of ownership (TCO) is very high as it motivates questionable specification practices, which in turn motivate change prevention and other poor behaviors by the development team.
  • Overly long estimation efforts in an effort to get the “right answer” often prove to be far more costly than the actual benefit provided.
  • Beware of misguided desires for “accurate” or “consistent” estimates which prove costly to produce in practice yet don’t improve the decision making ability of senior management to any great extent
Educated guess by experienced individual
  • Very quick and inexpensive way to get a reasonable estimate.
  • Requires that you have an experienced person involved with your project (if you don’t, then you should consider this a serious risk).
  • Explicitly reveals to stakeholders that estimating is often more art than science.
  • Some stakeholders may be uncomfortable with the fact that you’re guessing.
  • Beware of estimates by people who are inexperienced with the platform or domain or who do not know the abilities of team.

 

There are several reasons why DAD’s goal-driven approach is important:

  1. It makes your choices explicit.  If your team wants to truly own your process then it first needs to know what choices are available to it.
  2. It makes it clear that practices are contextual in nature.  No practice is perfect for all situations, every single one has advantages and disadvantages.  Are you choosing the ones that are most appropriate for your situation?
  3. You can have more coherent discussions of the trade-offs that you’re making.  We have endless religious battles in the IT industry around process-related topics.  This often happens because people choose to focus on the benefits of their favorite practices and to downplay the disadvantages (or worse yet are oblivious to them).  To help your team move to more effective practices it’s important to recognize the trade-offs involved with each, to then discuss them rationally, and then decide to adopt the strategy that is most viable given your situation.  Note that this doesn’t necessarily mean that you’re going to adopt the best practice from the start, but that instead you’ll adopt the best one that you can right now.  Later on, perhaps as the result of a retrospective, you’ll decide to make improvements to your approach (in the case of process factors where the strategies are ranked by effectiveness, your team may choose to adopt a strategy higher in the list).
  4. Improves the effectiveness of retrospectives.  During a retrospective it is fairly straightforward to identify potential problems that you’re facing, what isn’t as easy is identifying potential solutions.  You can improve retrospectives by having these process goal diagrams available to you to suggest potential strategies that you should considering adopting.
  5. You can avoid reinventing existing practices.  Many very smart and very experienced people have found ways to deal with the same challenges that you’re facing today.  Furthermore, many of them have shared these strategies publicly.  If you don’t know that these strategies exist you are at risk of wasting time reinventing them, time that could be better spent adding real value to your organization.
  6. It enables scaling.  Teams in different situation will make different process decisions.  Although teams at scale, perhaps they are large teams or geographically distributed, will follow many of the same practices as small co-located teams they will also adopt a few strategies that make sense for them given their situation.   DAD’s process goals provide the insight that teams need to understand how they can address the challenges associated with the scaling factors that they face.

For a more detailed discussion about the challenges around “best practices”, you may find the article Questioning Best Practices to be an interesting read.  The New Deal for Software Development provides some interesting insights as well.

Posted by Scott Ambler on: March 05, 2015 05:16 AM | Permalink | Comments (0)

DevOps Strategies: Operations

DevOps Practices - Operations

There are several technical strategies that support the operational aspects of DevOps:

  1. Solution monitoring.  As the name suggests, this is the operational practice of monitoring running solutions and applications once they are in production. Technology infrastructure platforms such as operating systems, application servers, and communication services often provide monitoring capabilities that can be leveraged by monitoring tools (such as Microsoft Management Console, IBM Tivoli Monitoring, and jManage). However, for monitoring application-specific functionality, such as what user interface (UI) features are being used by given types of users, instrumentation that is compliant with your organization’s monitoring infrastructure will need to be built into the applications. Development teams need to be aware of this operational requirement or, better yet, have access to a toolkit that makes it straightforward to provide such instrumentation.
  2. Standard platforms. Software development practices, such as continuous deployment and initial architecture envisioning, are enabled by consistency within your operational infrastructure. It is much easier to deploy to a handful of standard hardware configurations than it is to a myriad of unique ones. It is easier to deploy when there are consistent versions of infrastructure software (e.g. operating systems, databases, middleware, and so on) deployed across your environment. For example, all instances of your Oracle DB are 11.2.1.3, you don’t have 11.2.0.0, 12.0.1.0, and 11.2.1.3 installed in various places. Furthermore, it is much easier to make architecture decisions when there is consistency of infrastructure software packages in the first place. For example you standardize on Linuz for your server operating system, you don’t also have Windows, z/OS and others also in production (and if you do you’re actively retiring them).
  3. Deployment testing. After a solution, or an update to a component of your operational infrastructure, has been deployed you should run a quick set of tests to verify that the deployment was successful. Were the right versions of the files installed where they need to be? And were they deployed to all appropriate servers? Were database transformations applied successfully? Did the appropriate announcements, if any, get sent out? Did the overall deployment process run within the desired time frame?
  4. Automated deployment.  Deployments should be automated, not manual. This increases the consistency of your deployments and supports the practice of continuous deployment. Part of your automation effort should be to support both self-recovery and self-testing as native aspects of your deployment strategy.
  5. Support environments. Anyone doing solution support, even if it is the development team itself, is likely to need an environment in which they can reproduce problems that end users experience. There are several options available to you:
    • Production. In some cases your production environment is sufficient, although many regulatory regimes, particularly life-critical and financial-critical ones, will not allow this.
    • Pre-production test sandbox. Some support teams will find that they can use their pre-production test environment to try to simulate production problems. The advantage is that you don’t put production at risk when trying to reproduce problems, the disadvantage is that you the test environment will be different than production and as a result you may not be able to simulate all reported problems.
    • Support sandbox. Some organizations choose to have a specific environment set up to enable support staff to simulate production problems. This strategy has the same tradeoffs as using a pre-production test sandbox plus the additional cost and maintenance associated with yet another environment.

In the next blog posting in this DevOps series we will explore solution support strategies.

Posted by Scott Ambler on: February 19, 2015 04:59 AM | Permalink | Comments (0)

DevOps Strategies: General

Categories: DAD discussions, DevOps, DevOps

DevOps Practices

In a previous blog posting we overviewed the concept of Disciplined DevOps, which is the streamlining of IT solution development and IT operations activities, as well as supporting enterprise activities.  In this blog posting we begin to overview strategies that support DevOps.  This posting overviews general strategies, and future postings will describe development, operations, release management, data management, and enterprise architecture strategies.

There are several “general” strategies that support DevOps:

  1. Collaborative work.  A fundamental philosophy of DevOps is that developers, operations staff, and support people must work closely together on a regular basis. An implication is that they must see one other as important stakeholders and actively seek to work together. A common practice within the agile community is “onsite customer,” adopted from Extreme Programming (XP), which motivates agile developers to work closely with the business. Disciplined agilists take this one step further with the practice of active stakeholder participation, which says that developers should work closely with all of their stakeholders, including operations and support staff–not just business stakeholders. This is a two-way street: Operations and support staff must also be willing to work closely with developers.
  2. Automated dashboards. The practice of using automated dashboards is called IT intelligence, effectively the application of business intelligence (BI) strategies for IT. There are two aspects to this, development intelligence and operational intelligence. Development intelligence requires the use of development tools that are instrumented to generate metrics; for example, your configuration management (CM) tools already record who checked in what and when they did it. Continuous integration tools could similarly record when a build occurred, how many tests ran, how long the tests ran, whether the build was successful, how many tests we successful, and so on. This sort of raw data can then be analyzed and displayed in automated dashboards. Operational intelligence is an aspect of application monitoring discussed previously. With automated dashboards, an organization’s overall metrics overhead can be dramatically reduced (although not completely eliminated because not everything can be automated). Automated dashboards provide real-time insight to an organization’s governance teams.
  3. Integrated configuration management. With an integrated approach to configuration management (CM), development teams not only apply CM at the solution level as is customary, they also consider production configuration issues between their solution and the rest of your organization’s infrastructure. This can be a major change for some developers because they’re often used to thinking about CM only in terms of the solution they are currently working on. In a DevOps environment, developers need to be enterprise aware and look at the bigger picture. How will their solution work with and take advantage of other assets in production? Will other assets leverage the solution being developed? The implication is that development teams will need to understand, and manage, the full range of dependencies for their product. Integrated configuration management enables operations staff to understand the potential impact of a new release, thereby making it easy to decide when to allow the new release to occur.
  4. Integrated change management.  From an IT perspective, change management is the act of ensuring successful and meaningful evolution of the IT infrastructure to better support the overall organization. This is tricky enough at a project-team level because many technologies, and even versions of similar technologies, will be used in the development of a single solution. Because DevOps brings the enterprise-level issues associated with operations into the mix, an integrated change management strategy can be far more complex, due to the need to consider a large number of solutions running and interacting in production simultaneously. With integrated change management, development teams must work closely with operations teams to understand the implications of any technology changes at an organization level. This approach depends on the earlier practices of active stakeholder participation, integrated configuration management, and automated testing.
  5. Training, education, and mentoring.  As you would expect, people will need help to learn and adopt your DevOps strategies.
  6. Continuous improvement.  Disciplined agile teams strive to learn from their experiences as well as from others so that they can continuously improve the way that they work together, including how they approach DevOps.
  7. One team.  An important aspect of the DevOps mindset is shifting away from a “them and us mindset” to an “us mindset.”  We all work together as a single, streamlined team.  An extreme form of this is the “you build it, you run it” philosophy where there are no separate development, operations, data administration teams but instead product teams who are responsible for the entire lifecycle of a product.

Our next blog posting in this series will overview development-oriented strategies.

Posted by Scott Ambler on: January 27, 2015 04:32 PM | Permalink | Comments (0)

Disciplined Agile Portfolio Management

We recently published an article describing how the Disciplined Agile Delivery (DAD) framework is being extended to cover Portfolio Management activities.  The following diagram depicts the DAD goal diagram for Portfolio Management, and as you would expect your organizations has a range of options to choose from.  In this diagram we use the term endeavor to refer to a project, product, or experiment.

Disciplined Agile Portfolio Management

The article describes each one of these process factors – Identify Endeavors, Explore Potential Endeavors, and so on – in greater detail.  A future version of the article will describe the strategies/practices associated with each factor.  We have also included a high-level workflow diagram, see below, to overview from the point of view of the Portfolio Management process blade how it fits into the rest of the Disciplined Agile (DA) toolkit.

Disciplined Agile Portfolio Management

An interesting aspect of the flow diagram is that it shows the relationship of Portfolio Management to other IT Plan activities.  For example, Enterprise Architecture provides a Technology Roadmap and Product Management provides a Business Roadmap and an indication of stakeholder priorities – all critical information that are inputs into the efforts around prioritizing potential endeavors.  The point is that this diagram reflects workflow, not organizational design.  For example, in some companies there is no Product Management team, instead responsibility for the Product Management activities are spread out amongst several teams.  A common strategy is to have the Portfolio Management team responsible for understanding stakeholder priorities and the Enterprise Architecture team responsible for the business roadmap.  Another approach would be to have the Portfolio Management team subsume both of those activities.  A third approach would be to have three teams, one for each of Enterprise Architecture, Product Management, and Portfolio Management.  Different organizations will make different organizational design decisions, and of course these decisions will evolve over time.  As always, context counts.

I hope that you find the more detailed Portfolio Management article to be of interest.

Posted by Scott Ambler on: January 25, 2015 09:16 PM | Permalink | Comments (0)
ADVERTISEMENTS

"If you pick up a starving dog and make him prosperous, he will not bite you. This is the principal difference between a dog and a man."

- Mark Twain

ADVERTISEMENT

Sponsors