Project Management

Disciplined Agile

by , , , , , ,
This blog contains details about various aspects of PMI's Disciplined Agile (DA) tool kit, including new and upcoming topics.

About this Blog

RSS

View Posts By:

Tatsiana Balshakova
Mark Lines
Mike Griffiths
James Trott
Bjorn Gustafsson
Curtis Hibbs
Scott Ambler

Past Contributors:

Joshua Barnes
Michael Richardson
Daniel Gagnon
Valentin Tudor Mocanu
Kashmir Birk
Glen Little
Klaus Boedker

Recent Posts

DA 5.6 is released

Disciplined Agile 5.5 Released

Choose Your WoW! Second Edition Is Now Available

Requisite Agility applied in Project Management

Disciplined Agile and PMBoK Guide 7th Edition

Categories

#ChoiceIsGood, #ChooseYourWoW, #ConsumableSolution, #ContinuousImprovement, #CoreAgilePractices, #experiment, #Experimentation, #GuidedContinuousImprovement, #Kaizen, #LifeCycles, #ProcessImprovement, #TealOrganizations, Adoption, agile, agile adoption, Agile Alliance, Agile Business Analyst, Agile certification, agile data, agile governance, agile lifecycle, agile metrics, agile principles, agile transformation, Agile2018, Agile2019, Agile20Reflect, AgileData, Analogy, announcement, Architecture, architecture, architecture owner, Articles and publications, Asset Management, Atari, Backlog, Barclays, being agile, benefits, bi, blades, book, Branching strategies, Browser, Business Agility, business intelligence, business operations, capex, Case Study, Certification, certification, charity, Choose your WoW, CMMI, cmmi, Coaching, Collaboration, Communications Management, Compliance, Compliancy, Conference, Construction, Construction phase, Context, Continuous Improvement, coordination, COVID-19, Culture, culture, Cutter, DA, DAD, DAD Book, DAD discussions, DAD press, DAD roles, DAD supporters, DAD webcast, DADay2019, Data Management, database, dependencies, Deployment, Development Strategies, DevOps, disaster, Discipline, discipline, Disciplined Agile, disciplined agile delivery, disciplined agile delivery blog, Disciplined Agile Enterprise, disciplined devops, Documentation, Domain complexity, dw, DW/BI, Energy Healing, Enterprise Agile, Enterprise Architecture, Enterprise Awareness, enterprise awareness, Essence, estimation, Evolving DA, Executive, Experiment, facilitation, FailureBow, feedback-cycle, finance, Financial, FLEX, Flow, foundation layer, Funding, GCI, GDD, Geographic Distribution, gladwell, global development, Goal-Driven, goal-driven, goals, Governance, GQM, Guideline, Hybrid, Improvement, inception, Inception phase, India, information technology, infosec, Introduction, iterations, Kanban, large teams, layer, lean, Lean Startup, learning, Legal Project Management, LeSS, Lifecycle, lifecycle, Manifesto, mark lines, marketing, MBI, Metaphor, Metrics, metrics, mindset, Miscellaneous, MVP, News, News and events, Non-Functional Requirements, non-functional requirements, Non-solo development, offshoring, Operations, opex, Organization, Outsourcing, outsourcing, paired programming, pairing, paper, People, People Management, phases, Philosophies, Planning, PMBoK, PMI, PMI and DA, PMI Chapter, Portfolio Management, post-format-quote, Practices, practices, Principle, Process, process improvement, process tailoring, Product Management, product owner, Product Owners, productivity, Program Management, Project Management, project-initiation, Promise, Quality, quality, rational unified process, Refactoring, Reiki, Release Management, release management, Remote Training, Remote Work, repeatability, requirements, Requirements Management, research&development, responsibilities, retrospectives, Reuse, Reuse Engineering, ride for heart, rights, Risk Management, Risk Management, Risk management, Roles, RUP, SAFe, sales, Scaling, scaling, scaling agile, Scheduled Workshops, SCM, scorecard, Scrum, ScrumMaster, SDLC, Security, security, self-organization, SEMAT, serial, skill, solutions software consumable shippable, Stakeholder Management, strategy, Support, Surveys, Teal organizations, team development, Team Lead, team lead, Teams, Technical Debt, Teleconferencing, Terminology, terraforming, test strategy, testing, time tracking, Tool kit, Toolkit, tools, traditional, Transformation, Transition iteration, transition phase, Uncategorized, Upmentors, Using PMI Standards, value stream, velocity, vendor management, Virtual Training, Workflow, workflow, workspaces

Date

Viewing Posts by Scott Ambler

Disciplined Agile Terminology

linkedin twitter facebook Request to reuse this  

Argument

This brief article explains our thinking around our terminology choices in the Disciplined Agile (DA) toolkit. It overviews the terminology principles that we follow, discusses why Scrum terminology isn’t appropriate, and maps common Scrum terms to DA terms.

 

Our Principles Around Terminology

The following three principles drive our terminology decisions:

  1. Terms must be clear. If you need to explain the term, it likely isn’t the best. For example, how many times have you had to explain what a Scrum meeting is? Call it a coordination meeting instead, and people have a much better idea of what’s going on.
  2. Terms must be method neutral. Every team is unique and owns its own process. Part of owning your process is choosing the overall method, or lifecycle, that you’re following. Because the DA toolkit is a hybrid that leverages a variety of methods, were we to adopt one method’s terminology over another it would only make sense for people following that lifecycle. For example, Scrum terminology makes sense if you’re following the Scrum-based Agile/Basic lifecycle but not the Lean Continuous Delivery lifecycle.
  3. Terms should already be in use elsewhere. We are not in the business of creating new terms when existing ones are perfectly fine.

 

The Problem with Scrum Terminology

Many people ask us why we don’t simply use Scrum terminology. We originally wanted to, because that would be the easy thing to do, but we quickly realized that Scrum terminology just doesn’t get the job done for three reasons:

  1. It doesn’t apply in all situations. For example, the term “sprint retrospective” doesn’t really make sense when you’re following a lean lifecycle that doesn’t have the concept of sprints/iterations. Furthermore, it breaks principle #3 above in that the Scrum folks tacked “sprint “onto the front of the existing term “retrospective” to brand it with Scrum marketing.
  2. It was motivated by marketing reasons. The Scrum originators purposely chose unusual terms such as sprint, Scrum Master (later concatenated to ScrumMaster), and Scrum meeting to signal to people that Scrum was different. Well, in DA we’re purposely choosing pragmatic terminology to signal to people that it’s time to up our game as software professionals.
  3. It reflects 1990s thinking. There’s nothing wrong with that per se, other than the fact that we have learned a lot the following decades that we can apply.

 

Mapping Scrum to Disciplined Agile Terms

The following table maps common Scrum terms to the terms that we prefer in DA. As you can see, the mapping is very straightforward.

 

Scrum Term DA Term DA Source Observations
Backlog refinement/grooming Look-ahead modeling
  • “Modeling” is common IT terminology.
  • “Look-ahead modeling” is an existing Agile Modeling practice
  • Not all teams have backlogs.
  • The term isn’t clear (one reason why it evolved from backlog grooming to backlog refinement a few years ago)
Mapping Modeling
  • Common IT terminology
  • Agilists really need to get over their cultural issues around modeling and documentation
  • There is a wealth of material about effective modeling strategies that many agilists are unaware of because they search on terms such as mapping or grooming instead of modeling
Scrum Master Team Lead
  • Common IT terminology
  • Only Scrum teams have Scrum Masters
  • The term “Scrum Master” isn’t descriptive of what someone in that role does
  • The responsibilities of a Team Lead are a bit more robust than those of a Scrum Master, so this mapping isn’t perfect
Scrum meeting Coordination meeting
  • Common terminology
  • Coordination meeting is a much clearer term
Sprint Iteration
  • Iteration is used as a term in XP, Agile Modeling, Unified Process and many others
  • The term sprint is ok, but it doesn’t reflect the agile principle of maintaining a steady pace (you don’t sprint through a long race)
Sprint demo Demo
  • Common IT terminology
  • You can hold a demo at any time, not just at the end of a sprint
Sprint Retrospective Retrospective
  • Original term for the technique
  • You can hold a retrospective at any time, not just at the end of a sprint

 

Parting Thoughts

There is no standard terminology in the agile world, nor will their ever be. Your team, as part of owning your process, will need to decide which terms they prefer to use. We’ve seen many DA teams choose to use Scrum terminology (e.g. sprint instead of iteration) because they originally started with Scrum and that’s what they’re familiar with. That’s their decision and as always our advice is for a team to do what they believe to be right for the situation that they find themselves in.

Posted by Scott Ambler on: September 05, 2016 07:29 AM | Permalink | Comments (0)

How to Choose an Agile Release Cadence

linkedin twitter facebook Request to reuse this  

Metronome

One of the things that a delivery team needs to do, often in collaboration with product management, is choose the release cadence of their product. This is an important aspect of, you guessed it, release planning.  Your release cadence defines how often you release your solution both internally and externally into production (or the marketplace). The issue is how to determine how often the product should be released into production. In this blog we explore:

  1. Where are you deploying to?
  2. What affects release cadence?
  3. What release cadence choices do you have?
  4. What do we recommend?

 

Where Are You Deploying?

There are several target environments that you might choose to deploy to. These environments include:

  1. Demo environment(s). Many teams maintain a demo environment for their solution so that their stakeholders can see what has been developed to date at their leisure. Demo environments support transparency with your stakeholders, reduce the number of “one-off” requests by stakeholders for demos (because they can simply see the solution for themselves), and of course they provide a stable environment in which your teams can run demos.
  2. Testing environment(s). Many teams have their own testing environments, or they work with independent test teams with their own testing environments, or both. You should strive to test as often and early as possible, an implication being that want to deploy into your test environment(s) as often as you possibly can.
  3. Production/marketplace. Some teams will release their solutions into their production environments (or to someone else’s cloud) where end users will use the systems. In the case of commercial software companies they will release their solutions into the marketplace where they are then sold to customers. Throughout this blog whenever we use the term production we also mean the marketplace.

For the sake of terminology, deploying into demo or testing environments are often referred to as internal releases and into production as an external release.

 

What Affects Release Cadence?

There are several factors that affect the choice of release cadence:

  1. Stakeholder needs. How often do your stakeholders, and in particular your end users, want your solution to be released? This can be a difficult issue because very often your stakeholders might not be able to perceive what is appropriate for them. We’ve seen stakeholders ask for quarterly releases, be delighted when then get monthly releases, and then start asking for weekly releases once they realize the potential of modern agile strategies.
  2. Stakeholder capability to accept change. We like to think that more often is better, and in the vast majority of situations it is. As difficult to believe as this may seem, at the far extreme we’ve also seen some systems where the natural release cadence is once every three or four years because that’s the rate at which stakeholders are able to accept change. In this case the product was a transaction processing (TP) system infrastructure product, but we’ve heard similar stories about major database management systems (DBMSs) products too. Granted, a release cadence this long is very rare but it does happen in a small number of situations. Far more common is the mistaken belief by IT professionals that their stakeholders are unwilling or unable to accept shorter release cycles. We’ve seen numerous organizations where the IT people tell us that their stakeholders can’t handle anything more regular than a quarterly or bi-annual release, yet these same stakeholders regularly use commercial software that is updated several times a week.
  3. Your organizational culture. Some organizations, particularly those with an existing traditional release management team, often have release cultures that lean towards larger and less frequent releases. These organizations often have significant investments in legacy systems and insufficient investments in automated regression tests/checks. As a result releasing solutions into production tend to be seen as a risky endeavour. At the other extreme we’ve seen companies with more of a continuous delivery mindset that have a “release as swiftly and often as you can” culture. These organizations have typically invested heavily in code quality, automated regression testing, and automated deployment thus making deployment a simple and virtually risk-free effort.
  4. The team’s ability to deliver. Of course a primary determinant of your release cadence will be how often you’re able to actually produce a potentially consumable solution. This is affected by the skills of your team members, your ability to collaborate, your ability to vertically slice functionality into small features, and your delivery infrastructure.
  5. Your delivery infrastructure. How easy it is to release a potentially consumable solution into production is determined in part by your technical environment. This includes the extent of your automated regression tests, your automated deployment scripts, and your capability to monitor production. In general, the greater the level of automation the more often you can release.
  6. Your solution architecture. Is your solution architected to be released incrementally? Is it possible to enable/disable functionality at a granular level (perhaps via feature toggles or a similar technique)?
  7. The cost/risk to release.   Cost and risk tend to go hand-in-hand when it comes to releasing solutions into production. This is because the more manual your release/deployment processes the more expensive they become and the more likely there are to be problems somewhere in the process. Conversely, the more you automate the overall deployment effort the cheaper it is to deploy and the less risky it becomes as you’re more likely to run into, and then automate a solution to, deployment problems. The less expensive and less risky it is to release your solution the more viable it becomes to release more often.
  8. Release cadence of other teams. Like it or not your team is likely dependent on the work of other teams. For example you may need web services being built by another team, and you can’t fully release your solution until those web services are available in production.  We’ve written detailed articles about how to manage dependencies between agile/lean and traditional teamsdependencies between agile teams, and dependencies between agile and lean teams.

 

Release Cadence Choices

Table 1 lists many common release cadences, from more than annual to several times a day. It also lists the potential tradeoffs of each approach and indicates when you may want to adopt each one.

Table 1. Comparing external release cadence options.

Strategy Potential Advantages Potential Disadvantages When to Apply
Many times a day Enables very short time to market

 

Enables teams to adapt quickly to changing stakeholder needs

Enables granular release of functionality

Requires extensive continuous integration (CI) and continuous deployment (CD) automation

 

Requires high discipline to maintain quality

Your solution architecture must support toggling of features to enable deployment of larger functions as a collection of smaller features

Effective for high-use systems, particularly those used by external customers in highly-competitive environments
Daily Same as above

 

Provides a regular (daily) release cadence that is predictable

Same as above Same as above
Weekly Provides a regular (weekly) release cadence that is predictable

 

Enables quick time to market and responsiveness to changing needs

Same as above Effective for high-use solutions, particularly e-commerce or BI/reporting systems

 

Appropriate for teams following the Lean lifecycle

Monthly Provides a regular (monthly) release cadence that is predictable

 

Enables quick time to market and responsiveness to changing needs

Requires extensive continuous integration (CI) and continuous deployment (CD) automation

 

Requires high discipline to maintain quality

Effective for medium-priority solutions

 

Appropriate for teams following the Agile/Basic lifecycle with one-week iterations or the Lean lifecycle

Quarterly Provides a regular (quarterly) release cadence that is predictable

 

Enables quick time to market and responsiveness to changing needs

Enables simpler requirements management practices (compared with longer release cadences) due to lower impact of a feature moving to the next release

This is a major milestone for teams moving towards an “advanced” lean-agile strategy as it motivates greater discipline.

Requires continuous integration (CI)

 

Requires automated deployment strategies

Effective for medium-priority solutions

 

Appropriate for teams following the Agile/Basic lifecycle with one or two week iterations

Variable Works well with a project mindset (although that’s questionable in and of itself) Teams need to be able to judge when their work reaches the minimally marketable release (MMR) stage and the business value added exceeds cost of transition. This decision point is captured in the DAD project lifecycles by the “sufficient functionality” milestone

 

Politics can hamper this decision point. You should put an upper limit on the acceptable time between release

Project teams

 

Stable teams assigned large “projects”

Bi-annual Good starting point for teams new to agile who are currently working on traditional projects with longer release cadences because it motivates adoption of disciplined strategies Can be difficult for stakeholders who are used to less frequent releases The team may need significant agile coaching as they will run into many of the “but we’re different and that agile stuff can’t possibly work here” type of problems
Annual Provides a regular (annual) release cadence that is predictable

 

 

Very risky, the team is likely to miss their date

 

Requires internal releases to obtain feedback

The deployment has likely become high risk because you do it so infrequently (self fulfilling problem)

Appropriate for low priority systems or for high-risk deployments (note that the deployments may have become high-risk because you do them so infrequently)
More than annual   See annual

 

 

See annual

 

This is common for infrastructure systems, such as a database or transaction managers, that have many other systems highly dependent upon them

 

Our Recommendations

When it comes to releasing your solution, we have several recommendations for you to consider:

  1. Automate, automate, automate. The more you have automated, the lower the cost of deployment and the lower the risk. This enables you to release more often with confidence.
  2. Release internally very often. This is your opportunity to get good at releasing your solution, at squeezing out the cost and the risk.
  3. Release externally as often as possible. The faster and more often you can release into production the more competitive your organization will be.
  4. Always look for ways to release more often. Impressed with your ability to release once a month? Aim for bi-weekly. You’ve now releasing bi-weekly? What’s stopping you from releasing weekly? Weekly releases? Meh! Release daily! Your team is releasing daily grandpa? How about automatically releasing many times a day every time you have a working build?

 

Further Reading

Posted by Scott Ambler on: August 19, 2016 07:59 AM | Permalink | Comments (0)

Why is Disciplined Agile So Complicated?

linkedin twitter facebook Request to reuse this  

Confused

We often hear the refrain that Disciplined Agile (DA) is too complicated, that it needs to be simpler. Believe me, we’re all for simplicity and we do everything that we can to make DA as simple as possible. However, to be fair, we often find that the people complaining about this are often coming at the situation from a different point of view than we are.

With this blog we work through how why the complications encompassed by DA reflects the reality that we face in modern enterprises. To do this let’s explore why DA may appear to be complicated at first. This blog explores the following issues:

  1. There is a natural complications in what we do
  2. Enterprise solution delivery is inherently complicated
  3. Enterprise IT is inherently complicated
  4. Choice is good, but having choices adds complications
  5. People underestimate the inherent complications of the familiar
  6. DA purposefully includes some questionable choices
  7. Most people prefer to focus on a small part of the overall process
  8. What would you take away?
  9. What would you like to add?

There is a Natural Complications in What We Do

Software development by itself is complicated: we need to work closely with stakeholders to understand their needs, we need to architect and design solutions, we need to build or configure these solutions, we need to test them, we need to deploy them, we need to organize this effort somehow, and many other things. There’s clearly a bit of natural complexity in software development. Furthermore, the overall IT process surrounding software development adds even more complications to operate, manage, and govern our overall IT ecosystem.

This is why we get paid what we get paid – if it was easy, organizations could find a multitude of lower-skilled, and lower-paid, people to do this work.

Enterprise Solution Delivery is Inherently Complicated

Complex process

The Disciplined Agile Delivery (DAD) portion of the DA toolkit addresses solution delivery in enterprise-class organizations. Sometimes people who have only worked in startups or small companies don’t appreciate the realities of enterprise-class development. A small company that has been in business for a few years is very different than a large organization that has several decades of legacy systems, legacy processes, and legacy thinking in place. Adopting agile and lean ways of working when you effectively have a clean slate to start from is a very different proposition than working in an organization where you need to respect and evolve the existing legacy ecosystem.

But don’t other, simpler methods such as Scrum work in these settings? Yes, they potentially do, but they address a much smaller scope than does DAD. Scrum is less complicated because it only looks at a very small portion, likely not even 5%, of the software delivery process. The focus of Scrum is on a few collaboration patterns for organizing small software construction teams and for managing changing requirements. It purposefully does not address the full project delivery lifecycle from beginning to end. Nor does Scrum address technical practices for architecture, design, testing, programming, modeling, documentation, deployment, and many other aspects of solution delivery. DAD, on the other hand, purposely addresses all of these aspects of software delivery from beginning to end in a streamlined manner. And Scrum only describes a single way of working, what DAD calls the Agile/Basic lifecycle, whereas DAD includes six lifecycles (Agile, Lean, Continuous Delivery: Agile. Continuous Delivery: Lean, Program, and Exploratory/Lean Start Up) to choose from to address a great range of needs. So yes, as a toolkit DAD is much more complex than Scrum because DAD answers the multitude of questions that Scrum leaves up to you.

Interestingly, from the point of view of enterprise agile transformation, DAD is a less complicated solution to adopt because it’s already done the “heavy lifting” for you on the process side of things.  We wrote a blog a few years ago about how it’s easier to right-size your process by starting in the middle with something like DAD than it is to start at the “simple bottom” with Scrum.

Enterprise IT is Inherently Complicated

Disciplined Agile IT (DAIT) extends DAD (and Disciplined DevOps) to look at all of IT. The workflow of a Disciplined Agile IT department is depicted in the following diagram, and that diagram is merely a high-level overview. Many of the challenges that developers like to complain about – dysfunctional governance, ineffective enterprise architecture, slow data management, non-existent reuse, questionable HR processes, and many more – DA chooses to explicitly address. Because DA deals with the much wider range of enterprise IT issues than just software development it encompasses more complexity than many developers may be comfortable with, or even aware of.

DA addresses this bigger picture because our philosophy is to provide coherent, pragmatic advice to help organizations address the challenges that they face in improving their processes. In fact, the DA toolkit appears to be the only source of information that addresses enterprise agile IT in a coherent and comprehensive manner.   Without a holistic vision such as this an organization will end up development a piece-meal strategy where the individual parts may be effective on their own but do not integrate well as a whole. Not looking at the entire IT picture at once is the primary cause of organizational dysfunctional in your department today, with the DA toolkit you can avoid this problem as you evolve into an agile/lean way of working.  BTW, you can download a copy of the workflow diagram shown above from the Disciplined Agile posters page.

Choice is Good, But Having Choices Adds Complications

Disciplined Agile looks at a range of situations, giving you choices. Other methods and frameworks, such as LeSS and SAFe, prescribe a single way of working.   Certainly a small development team will work differently than a medium-sized development team which works differently than a large team. A team that is co-located will work differently than a team spread across a single building, than a team spread across several buildings in the same city, than a team spread across the world. A team working in a regulatory environment will work in a more sophisticated way than a team in a non-regulatory environment. A team where everyone works for your company will work differently than a team where some of the work is being outsourced. A team facing a simple domain problem will work differently than a team facing a complex problem. And so on. Although many people just want to be told what to do, they just want a single way of working given to them, that desire for prescription is a recipe for disaster in modern IT organizations. You have a range of teams facing a range of situations: One process size does not fit all.

Many choices

Having choices is definitely complicated. This is similar to “needing a new outfit”, going to the store, only to be overwhelmed by the myriad of choices presented to you.

Small number of choicesOne way to deal with the this is to divide and conquer. Recognize that deciding on an outfit that is right for you is actually a collection of simpler choices. For example, your need to choose the right shirt, the right pants, the right shoes, and so on. Of course you shouldn’t make these choices in isolation because your outfit needs to work together as a whole.  You either need to coordinate your decisions and effectively make them in parallel or if you prefer to make decisions one at a time then be prepared for your shirt choice to limit your follow-on clothing choices.

Let’s bring the conversation back to software process. In Disciplined Agile Delivery (DAD) we called out a collection of process goals that a team should fulfill in some way to be successful at agile solution delivery. These goals included exploring the initial scope, putting the team together, developing a consumable solution, coordinating activities, and deploying the solution into production to name a few. Each process goal is overviewed by a process goal diagram, the one for Explore Initial Scope follows below. Each goal is organized into a collection of process decision points, each decision point has options, and each option has advantages and disadvantages. Where one agile method might advise you to write a collection of epics and user stories to identify the initial scope, in DAD we say that you need to explore how people will use your solution (and writing epics and user stories are two of many options available to you), you may need to explore the user interface, you should consider to what extent (if any) the initial requirements should be documented, how you will go about eliciting requirements, and other important decisions. Ideally, if you want to be effective, then you want each team to own their own process and make the choices that are best for them given the situation that you face.

Back to the clothing analogy. Just like Scrum and SAFe prescribe one way of working, you could similarly take a prescriptive approach to shopping for new clothes. This is the equivalent of saying that you will be successful if you wear a red polo shirt, blue pants, blue socks, and black shoes. That may be great advice if you are working as a food server at your local restaurant chain but horrible advice if you need an outfit that is appropriate for a wedding, or for working in a formal office, or for going to the beach. When buying the right clothing you need first need to understand the context and then hopefully have several choices available to you. You want to be able to choose the right shirt for you, not the one prescribed to you. You want to choose a nice pair of pants or a skirt that complements that shirt, and so on. Yes, it’s much easier just to buy exactly the clothes that you’re told to, but it doesn’t give you as satisfying a result in most situations.

In Disciplined Agile we extended this choice-driven approach to IT-level activities such as Portfolio Management, Enterprise Architecture, Reuse Engineering, Data Management, and others. The following diagram is the process goal diagram for Enterprise Architecture, depicting the process decision points that your EA team should consider addressing and a range of options for each choice.

 

People Underestimate the Inherent Complications of the Familiar

Casual guy

When most people are asked what the gentleman in the picture is wearing, they’ll say a blue dress shirt and slacks. However, he’s also wearing shoes, socks, a belt, and underwear (we hope).   It is quite common for people to abstract away, to ignore, details that are familiar to them – the “shirt and slacks” answer abstracted away the details of the entire outfit. However, for someone to whom western clothing is new, this “simple outfit” appears quite complex to them until someone helps them to understand it.

The exact same thing happens in the software world. When asked what we do it’s easy to casually say that we work with stakeholders to understand what they want, we write some software, and then we deploy it. Of course the reality of what actually goes on is a lot  more complex than that.

We invite you to spend a few minutes and jot down all of the activities that your team does to actually build and deploy software in practice. Pretty long list, isn’t it? And for every strategy on that list, you’ve selected one of many options available to you.

DA Purposefully Includes Some Questionable Choices

Good and bad choicesWe all make choices in our diet. Some of those choices are good and some of those choices are rather bad for us. We know that we’re making poor choices yet we still choose to do so, perhaps because that’s the best we can do at the time or simply because that’s what we feel like eating right now.

It’s exactly the same when it comes to process – we have a range of choices and some of them are better for us than others. For example, consider the choices we have when it comes to the level of detail that we capture when exploring the initial scope. We could choose to: write a very detailed requirements specification; write a light specification (perhaps using index cards, whiteboard sketches, and a bit of overview documentation); write a simple bulleted list of goals; or not capture any requirements information at all. Many agile purists will say that writing a detailed requirement specification isn’t agile, and in the vast majority of situations it isn’t, although we have seen life-critical regulatory situations where it is appropriate.   But, what if you’re in a situation where detailed requirements specifications aren’t appropriate yet someone is demanding that you create one? Agile purists will likely get into religious arguments with this person: the traditionalist will argue that their way is best and the purist will argue that there way is best, but in the end the person with the most political clout will win. With a DAD-based we recognize that there is a range of options available to you, including writing detailed specifications. But we take it one step further and are clear about the advantages and disadvantages of each approach and when the approach is appropriate.   Now you can have more intelligent conversations and pragmatic about your process choices.   When you get into situations such as this you can go beyond the “but that isn’t agile” refrain to have coherent reasons that as to why one strategy is better for another strategy given the situation that you face. Choice is good.

Most People Focus on a Small Part of the Overall Process

Locally focused

Many people choose to focus on their part of the job, which is fair enough. But what about all of the other activities that your team performs from beginning to end? You may be a developer focused on software construction, but what about all of the initial inception work that occurred, such as gathering initial requirements, putting the team together, and getting funding for construction that occurred before you began your work? What about the activities that occur within other teams that interact with, often to support, your team? For example, there may be some enterprise architects responsible for evolving and supporting a technical roadmap for your organization, there may be an infrastructure team responsible for operating your technical environment, and there may be people responsible for people management/HR activities (such as seeing that you get paid). Disciplined Agile looks at the holistic picture, not just at your piece of the overall puzzle. Just because you’re not involved with data management, or portfolio management, or operations doesn’t mean that those activities, and the teams that perform them, aren’t important to the overall success of your organization.

What Would You Take Away?

Now that you understand the overall scope of DA, what would you remove? We’re asking this question from the point of view of:

  • The complexity faced by modern enterprises
  • That you need choices if you’re to actually own your process
  • That we’re considering the overall picture, not just your part of it

If you think Disciplined Agile (DA) contains something that it doesn’t need, please add a comment at the end of this blog.

What Would You Like to Add?

Ha! Every single time we’ve had this conversation with someone, and they were willing to consider the issues above, they’ve told us that we needed to include one or more techniques that they’ve found effective in practice. EVERY. SINGLE. TIME.  In short, the conversation goes from DA is too complex to DA is not complex enough.  Argh!

So, once again, please add a comment at the end of the blog for anything you think we’re missing.

A Few Parting Thoughts

Software development, IT in general, and in fact your organization is a complicated game with many moving parts that are evolving all of the time. It’s natural for us, as human beings, to want simple answers. But sometimes the simplest answer can still be quite complicated. Simplistic answers may be more attractive, and in the short term or in very narrow situations they may in fact be a good solution for you, but more often than not they’ll make your situation even worse.   In Disciplined Agile (DA) we actively strive to drive the complications out of the software process but at the same time we have the courage to look at the bigger picture and explicitly address the range of issues that modern enterprises face in practice. With DA we’re working together to make the process as simple and streamlined as possible.

Puzzle - canstockphoto18141553 Small

Posted by Scott Ambler on: August 15, 2016 08:25 AM | Permalink | Comments (0)

Independent Testing and Agile Teams

Categories: agile, testing, Scrum

linkedin twitter facebook Request to reuse this  

The majority of testing, and in simple situations all of it, is performed by an agile delivery team itself. This is because we strive to have cross-functional “whole teams” that have the capability and accountability to perform the activities of solution delivery from beginning to end.   For organizations new to agile this means that you embed testers on agile teams and for organizations experienced in agile that you’ve managed to motivate agile team members to gain testing skills over time (often via close collaboration with other people who already have those skills).

This blog is organized into the follow topics:

 

Parallel Independent Testing

The whole team approach to development where agile teams test to the best of the ability is a great start, but it isn’t sufficient in some scaling situations. In these situations, described below, you need to consider instituting a parallel independent test team that performs some of the more difficult or expensive forms of testing. As you can see in Figure 1, the basic idea is that on a regular basis the development team makes their working build available to the independent test team. This may happen several times an iteration – we’ve seen teams do so at the end of each iteration, on a nightly basis, as part of their continuous deployment (CD) strategy. How often this occurs needs to be negotiated between the two teams.

Figure 1. Independent testing with an agile team.

Independent agile testing

The goal of this independent testing effort is not to redo the confirmatory testing that is already being done by the development team, but instead to identify the defects that may have fallen through the cracks. The implication is that this independent test team does not need a detailed requirements speculation, although they may need updated release notes including a list of changes since the last time the development team sent them a build. Instead of testing against the specification, the independent testing effort typically focuses on:

  1. Investigative/exploratory testing. Confirmatory testing approaches, such as test-driven development (TDD), validate that you’ve implemented the requirements as they’ve been described to you. But what happens when requirements are missed? What happens when requirements are narrowly focused on a point-specific solution, but not the overall ecosystem? User stories are a great way to explore functional requirements but defects surrounding non-functional requirements such as security, usability, and performance have a tendency to be missed by teams new to this approach.
  2. Production readiness testing. This is sometimes called pre-production system integration testing. The system that you’re building must “play well” with the other systems currently in production when your system is released. To do this properly you must test against versions of other systems that are currently under development, implying that you need access to updated versions on a regular basis. This is fairly straightforward in small organizations, but if your organization has dozens, if not hundreds of IT teams underway it becomes overwhelming for individual development teams to gain such access. A more efficient approach is to have an independent test team be responsible for such enterprise-level system integration testing.
  3. Difficult testing. Many forms of testing require sophisticated skills and sometimes even expensive tooling. Security testing is a classic example.

 

Why Independent Testing?

There are several reasons why you should consider independent testing:

  1. Regulatory compliance. Disciplined agile teams that find themselves in strict regulatory compliance situations, typical in systems engineering and life critical environments, may need to perform independent testing by law. Having said that, only a portion of your testing efforts will need to be performed independently. We have yet to run into a regulation that says that all of your testing needs to be performed independently, although we have seen several organizations that choose to interpret the regulations that way.
  2. Production readiness testing is exponentially expensive in multi-team environments. Development teams may not have the resources required to perform effective production readiness testing, resources that from an economic point of view must be shared across multiple teams. For example, if there are 5 other development teams in addition to my own then chances are that each team can do the work required to integrate and test against the builds of the other teams. But what about if there’re ten other teams? Twenty? Fifty? It becomes exponentially expensive for each team to do this integration and testing work as the number of teams increases. The implication is that you will discover that you need an independent test team working in parallel to the development team(s) that addresses these sorts of issues.
  3. The average cost of fixing defects is exponentially expensive the longer you wait. For close to four decades Barry Boehm, and other researchers, have gathered data showing that the average cost of addressing a defect rises exponentially the longer it takes to fix. This is depicted in Figure 2. The implication is that we want to find defects as early as we can, in fact ideally we want to build quality in from the start and not inject the defects to begin with. In traditional environments we would have left some forms of testing, in particular system integration testing (SIT) and user acceptance testing (UAT) to the end for the convenience of the people doing the testing (“we need to have everything in place before we can test it”). This results in very expensive and risky defect repair. Parallel independent testing often proves to be a bit more complex for testers, at least at first, but results in much more economical repair efforts.
  4. Complex technical environments. When you’re working with multiple technologies, legacy systems, or legacy data sources the testing of your system can become very difficult. System integration testing (SIT), performance testing, load testing, and security testing become more complex and more important in these situations.
  5. Large or geographically distributed teams. Large agile teams or geographically distributed agile teams are often subdivided into smaller teams, and when this happens system integration testing of the overall system can become complex enough that a separate team should consider taking it on. In fact, SAFe prescribes this with their System Integration Team strategy (which is virtually identical to this strategy). Granted, the reason why you have such teams is because you’re facing either significant domain or technical complexity.
  6. Outsourcing. Teams that are organizationally distributed, for example when some of the work is being outsourced to another organization, will very likely want to perform independent testing to validate the work being performed by the other company(es).  Read this article about agile outsourcing strategies.

 

Figure 2: Average cost of change curve.

Average cost of change curve

 

Questionable Reasons to Adopt Independent Testing

We’ve run into a few rather poor excuses to justify independent testing over the years:

  1. Your testing/quality staff prefer to work in a traditional manner. They may insist on testing from a detailed requirements specification, testing that is better performed by the team. They may insist on waiting to test the entire solution once it’s ready, instead of incrementally testing the system while it’s being built (enabling much cheaper defect fixing as discussed earlier). They may insist on all testing being done independently, instead of embedding people with testing skills on solution delivery teams. All of these strategies are choices that reflect a traditional culture, not an agile one. The real solution is to overcome these cultural challenges and help them to gain the skills and mindset required to work in an agile manner.
  2. Testing is outsourced. Some organizations will choose to outsource their testing to an external organization that is focused on testing.

 

But That Isn’t Agile!

Bullshit! Please excuse my language. Disciplined agile is pragmatic, going beyond the limited approach promoted by many agile purists.  These purists will claim that you don’t need parallel independent testing, and in simple situations this is clearly true. The good news is that it’s incredibly easy to determine whether or not your independent testing effort is providing value: simply compare the likely impact of the defects/change stories being reported with the cost of doing the independent testing. In short, whole team testing works well for agile in the small, but for more complex systems and in some tactical scaling situations you need to be more sophisticated.

Posted by Scott Ambler on: August 02, 2016 07:07 AM | Permalink | Comments (0)

Lean Thinking Provides a Philosophical Foundation for Scaling Agile

Categories: agile, Scrum, Kanban, lean, scaling, mindset

linkedin twitter facebook Request to reuse this  

Little ideas add up

Lean thinking is important for scaling agile in several ways:

  1. Lean provides an explanation for why many of the agile practices work.  For example, Agile Modeling’s practices of light weight, initial requirements envisioning followed by iteration modeling and just-in-time (JIT) model storming work because they defer commitment to the last most responsible moment.  These practices also help to eliminate waste because you’re only modeling what needs to be built at the point in time that it needs to be built.
  2. Lean offers insight into strategies for improving your software process.  Lean principles such as optimizing the whole and delivering quickly motivate you to look beyond your existing specialized processes to explore how everything fits together and to streamline it.  Identifying and understanding the sources of waste in your IT processes can motivate you to improve the way that you work and thereby eliminate the waste.  The lean principle of building quality in
  3. Lean thinking provides a philosophical foundation for scaling agile approaches.  No methodology, process, procedure, or framework is ever complete.  Nor can they be because you can always capture more detail for a wider range of situations.  Because of this incompleteness you need a collection of higher-level principles or philosophies to guide people when their process/procedure/… proves to be incomplete for the situation that they face.  Lean thinking has proven to be a very good source of such philosophies, as do other sources (Steven Covey’s principles come to mind, as does the work of Peter Senge).
  4. Lean provides techniques for identifying waste.  Value stream mapping, a technique common within the lean community whereby you model a process and then identify how much time is spent on value-added work versus wait time, helps calculate overall time efficiency of what you’re doing.  Value stream maps are a straightforward way to illuminate your IT processes, providing insight into where significant problems exist.  I’ve created value stream maps with several customers around the world where we analyzed their existing processes which some of their more traditional staff believed worked well only to discover they had efficiency ratings of 20-30%.  You can’t fix problems that you are blind to.
Posted by Scott Ambler on: July 10, 2016 09:19 AM | Permalink | Comments (0)
ADVERTISEMENTS
ADVERTISEMENT

Sponsors