Project Management

Disciplined Agile

by , , , , , ,
#ChooseYourWoW | #ContinuousImprovement | #Kaizen | #ProcessImprovement | Adoption | agile | Agile certification | agile transformation | Analogy | Architecture | architecture | book | Business Agility | Certification | Choose your WoW | CMMI | Coaching | Collaboration | Compliancy | Configuration management | Construction phase | Context | Continuous Improvement | COVID-19 | Culture | culture | DAD | DAD discussions | DAD roles | Data Management | database | DevOps | Discipline | disciplined agile delivery | Documentation | DW/BI | Enterprise Agile | Enterprise Architecture | Enterprise Awareness | Essence | Evolving DA | Experiment | Financial | GDD | Geographic Distribution | global development | Goal-Driven | goal-driven | goals | Governance | Guideline | Hybrid | Improvement | inception | Inception phase | Kanban | Large Teams | Lean | Lifecycle | lifecycle | Metrics | mindset | News | News and events | Non-Functional Requirements | non-functional requirements | Operations | Outsourcing | People | Philosophies | Planning | PMI | PMI and DA | Portfolio Management | Practices | Principle | Process | process improvement | Product Management | Product Owners | Program Management | Project Management | Promise | quality | Release Management | Requirements | requirements | Reuse Engineering | Risk management | RUP | Scaling | scaling | scaling agile | Scrum | Support | Surveys | Teams | Technical Debt | Terminology | Testing | testing | Toolkit | Transformation | Workflow | show all posts

About this Blog

RSS

View Posts By:

Scott Ambler
Glen Little
Mark Lines
Valentin Mocanu
Daniel Gagnon
Michael Richardson
Joshua Barnes

Recent Posts

Failure Bow: Choosing Between Life Cycles Flowchart Update

Evolving Disciplined Agile: Guidelines of the DA Mindset

Evolving Disciplined Agile: Promises of the DA Mindset

Evolving Disciplined Agile: Principles of the DA Mindset

Evolving Disciplined Agile: The DA Mindset

Do Agile Teams Pay Down Technical Debt in Practice?

Categories: Technical Debt

Technical debt is “a concept in programming that reflects the extra development work that arises when code that is easy to implement in the short run is used instead of applying the best overall solution”. Technical debt can be compared to monetary debt in that if it is  not repaid, it can accumulate ‘interest’, making it harder to implement changes later on.  Important questions to ask are “How common is it for agile teams to run into technical debt in practice?” and “When they do run into technical debt, are they paying it down?”

The following diagram summarizes responses to our question from our 2016 Agility at Scale study around technical complexity.  As you can see 84% of agile teams are working with legacy functionality and 51% with legacy data sources (so yes, the vast majority of teams are working in environments that are likely to have some form of technical debt).  More importantly, 38% of teams are actively paying down functional technical debt and 32% are paying down data technical debt (48% are paying down one or the other).

Technical Debt in Practice

 

Related Posts

 

 

Posted by Scott Ambler on: March 13, 2017 05:01 PM | Permalink | Comments (0)

When Should You Assign Points to Defects?

Categories: Technical Debt, Testing

A common question that we get from teams who are new to agile is whether you should assign points (sizes) to defects.  From a Disciplined Agile point of view we know that the answer will vary depending on the context of the situation, or in other words “it depends.”   The really disciplined answer though is to say on what it depends, which is what we explore in this blog.

Rod Bray, a senior agile coach at Scott Ambler + Associates, suggests this principle:

Don’t reward people for shoddy work

An implication of this is that if a team is injecting defects, they’re producing shoddy work, then it makes sense that their velocity suffers when they have to spend time fixing those defects.  The decrease in velocity is a visible trigger to investigate, and address, the cause of the productivity loss.  But what if the “shoddy work” wasn’t caused by your team?  What if the “shoddy work” was previously acceptable to your stakeholders?  Hmmm… sounds like the context of the situation counts in this case.  The following flowchart works through common thinking around when to size a defect.  Note that this is just our suggestion, your team could choose to do something different.  Let’s work through common scenarios to understand whether defects should be sized or not.  These scenarios are:

  1. Is it existing technical debt?
  2. The defect is injected and found by the team
  3. The defect is found by independent testing
  4. The defect is found after the solution is released

 

When do you size defects

 

Do you size the repair of existing technical debt?

The quick answer is sometimes.  For the sake of discussion we’ll assume that this technical debt has existed for a long time, potentially years.  From a purist point of view technical debt may be injected at any time (this is obviously true) and as a result may only be seconds old.  In the case of technical debt that is very new, refer to the logic below.

The key issue is whether fixing the defect is going to be a substantial effort. This of course is subjective.  Is the work substantial if it’s less than an hour of effort?  Several hours?  Several days?  Several weeks?  This is a bar that your team will need to set.  Something that will impact your schedule is likely to be substantial (and something your Product Owner is likely going to want to prioritize so that it can be planned for appropriately).  Clearly a defect that takes several days or more to fix is substantial and one that takes less than an hour is likely not.  Something that takes several hours or a day or so is likely something you need to think about.

 

Do you size defects injected and by found by the team?

No, the exception being technical debt (see above).  This falls under the principle of not rewarding the team for shoddy work.

 

Do you size defects found by independent testing?

Some teams, particularly those working in regulatory environments or working in complex environments, may be supported by an independent test team.  An overview of the independent testing strategy is presented below.  With the exception of the defect being found in existing technical debt (see above), the defect should not be sized.  Once again, the principle described earlier holds.  Of course if you don’t have an independent test team supporting your team then obviously you can ignore this question.

Independent agile testing

 

Do you size defects found in production?

The answer is sometimes.  As you can see in the high-level lifecycle diagram below, the DA toolkit explicitly recognizes that change requests are often identified by end users of an existing solution.  These change requests are effectively new requirements that should be prioritized and worked on appropriately.  Many organizations will choose to distinguish between two types of change requests, defects and enhancements, so that they may be treated differently.  The issue is that defects are often tracked and, if the team is smart, they are analyzed to determine how they got past the team in the first place.  Additionally you may find that defects and enhancement requests are charged for differently (a topic for a future blog).

Disciplined Agile Lifecycle - High Level System

If you don’t distinguish between defects and enhancement requests then there’s nothing to do, you simply treat the change request as a new requirement and size it like you normally would.  But if you do distinguish between types then you need to think about it a bit.  If the defect is found during a warranty period then it shouldn’t be charged for. Sometimes, particularly when work is being outsourced, there is a warranty period of several weeks or even months after something is released where the development team is expected to fix defects for free.  In this case you likely wouldn’t size the work in line with the principle described earlier.  Once you’re out of the warranty period then it’s likely you want to assign points to it: The functionality in question passed testing and was accepted by your stakeholders, so they in effect have taken responsibility for it.

To summarize, the answer to the question “Should we assign points to a defect?” is “it depends.”  In this blog we explored in detail why this is the case and described when and when you wouldn’t want to assign points.

I’d like to thank Kristen Morton, Rod Bray, and Glen Little for their insights which they contributed to this blog.

Posted by Scott Ambler on: October 30, 2016 10:40 AM | Permalink | Comments (0)

Crushed By Technical Debt?

Categories: Technical Debt

 

Crushed by debt

On Wednesday October 21, 2015 the Disciplined Agile Consortium hosted a webcast entitled “Crushed by Technical Debt?” A PDF of the slides can be found on Slideshare and the recording of the webcast can be downloaded here from Youtube.  The webcast addressed the following issues:

  • What is technical debt?
  • How can we avoid technical debt?
  • How do we identify technical debt?
  • How do we fix technical debt?
  • When should we accept technical debt?
  • How can we fund technical debt?

As you can imagine we ran out of time to answer all of the questions that we received during the presentation. Below are answers to all of the questions we received, even the ones that were answered during the Q&A portion of the webcast. The questions, and their answers:

Additional topics:

Question: What’s your opinion regarding separating tech debt into separate stories?

This is one way of doing it. At one point during the presentation we discussed three strategies for how to fund paying down technical debt.  These three strategies are:

  1. Technical debt removal as regular investment. The idea is that developers automatically pay down technical debt, just like you would automatically clean up spilled juice in the middle of your kitchen floor, whenever you run into it.  In effect paying down technical debt becomes part of your culture.
  2. Stories to address technical debt. The idea is that you have requirement artifacts, such as technical stories, to remove a small amount of technical debt within your environment.
  3. Projects to address technical debt. With this approach you have specific projects to rewrite portions of an IT solution, or perhaps even to remove that solution completely.
Strategy Advantages Disadvantages
Regular investment (cultural)
  • Removal becomes an automatic part of your overall strategy
  • Often seen as an overhead and as a result technical debt removal is often underfunded
Technical stories
  • It is easy to plan and estimate the required effort
  • It is easy to deprioritize removal of technical debt
  • It becomes an occasional task instead of an automatic response
Debt removal projects
  • Enables organizations to directly fund the remove of large amounts of technical debt at once
  • Difficult to get support for these kinds of projects as they are big ticket items that don’t directly support creation of new functionality for the business

Question: How to convince Product Owners to address Technical Debt?

Product Owners (POs) are responsible for prioritizing the work. When technical debt is addressed via stories then the PO can easily deprioritize such stories.  The implication is that the team members, and in particular the Architecture Owner, must work closely with the PO to help them to understand the implications of technical debt and why it’s important to pay it down.

When technical debt is addressed automatically via regular investment, it is a built in overhead that affects your team’s overall velocity (negatively in the short term, but positively in the long term.

Question: What do you do when there are to many warnings thrown by your code analysis tools?

Regarding using code-analysis output as a metric: do you have any specific advice for dealing with the scenario with legacy code where it will initially overwhelm you with thousands of issues, from the trivial (i.e. spelling mistakes) to major : i.e. should one start by switching off the trivial warnings and deal with the major, or start fixing the big-numeric-count issues no matter how trivial just to get the numbers down?

It’s harsh to say, but the reason why you’re getting a lot of warnings is because the code sucks. When this is the case, it’s important to recognize the situation for what it is and get going on fixing the problems.

Initially I would turn off a lot of the lower priority checks so that the team could focus on the critical issues with their code. Once they refactor those away then I would turn on the next group of high-priority checks and so on until the problems are dealt with.  This could take a long time, so you’ll need the discipline to stay at it.  Perhaps start each retrospective with the question “How many code checks can we turn on this coming iteration?”

Question: Lack of (enough) clean code from the first time?

This is always a major frustration for people.   It is incredibly rare to be in a situation where clean code was written from the very beginning and then kept clean via techniques such as refactoring.   Instead most people find themselves swimming in legacy code, data sources, user interfaces, and so on that contain lots of technical debt.  Our advice is to accept:

  • The situation that you find yourself in
  • That the technical debt isn’t going to fix itself
  • It’s only going to get worse
  • You need to start paying down your technical debt now

If it makes you feel any better, and it likely won’t, the 2014 Software Development at Scale survey found that 92% of agile teams were facing some form of technical complexity.

Question: How we should measure technical debt?

There are many different ways to measure technical debt, all focused around solution quality. You may choose to measure code quality, perhaps in an automated fashion via code analysis tools, for example.  You may also choose to track defect trends over time, although that can easily become a measure of how good you are at testing instead of the quality of what you’re producing.

Question: How do I know if it’s better to fix technical debt or start all over from scratch?

This typically comes down to a gut feel based on your experience doing such work in the past.

Question: What does a negative score for technical debt awareness means? How should that be read & interpreted?

This is a reference to one of the charts from the 2015 Agile Cadences and Technical Debt Survey, see below , where senior business management had a negative score.  The range is from -10 (very unaware) to +10 (very aware).  So on average business management leaned towards being unaware of the implications of technical debt.  This is a problem seeing as the business needs to fund the removal of technical debt.

Technical Debt Awareness

 

Question: Is there any best practice/method to forecast Technical Debt?

Unless you really have your act together, the forecast is that technical debt is going to get worse over time.  Seriously though, that’s why you need to have a technical debt metrics program in place. In this case you want to track the trends over time, and simply extrapolate those trends to forecast the implications to your organization of technical debt.

Question: Can one architect span multiple teams? That is, is it a good idea to have the same person as the architect across multiple teams?

In Disciplined Agile we explicitly include the role of Architecture Owner (AO), which was adopted from the Agile Modeling methodology. This is effectively an agile solution/application architect who works in a collaborative, evolutionary, and facilitative manner.  Every delivery team should have someone in the AO role to help guide them through architectural issues.  Ideally each team has one person in this role and they are dedicated to the team full time, just like other team members.

Having said that, as we like to say “you go to war with the army that you’ve got.” For example, you may have ten agile teams but currently only four people with the skills to be an AO.  In this situation each AO is going to be a member of two or three teams.  As a result they will suffer a personal productivity loss from task switching and their teams will suffer a productivity loss from having to sometimes wait for them to be available.  This isn’t ideal but it’s the reality that many organizations face.  The solution is for the AOs to work collaboratively with other team members so as to start passing on their skills and knowledge to them, to help grow their fellow team members.

Question: Are the goal diagrams from both DAD 1.x and DAD 2.x available in a single document?

Great question. Sadly the quick answer is that there isn’t a print-based one at the current time.  Our focus has been on publishing to the web, ignoring the fact that we recently published the 104-page book Introduction to Disciplined Agile Delivery. We have published a landing page for the 22 Process Goal Diagrams for Disciplined Agile Delivery. We are also actively publishing the goal diagrams online for the IT process blades, such as Enterprise Architecture and Portfolio Management, as well.  The easiest way to access them is from the diagram on the Disciplined Agile Home Page (just click on the corresponding bubbles). This is a work in progress and we hope to have the process blades descriptions in place by December.

Having said that, we have been thinking about publishing some sort of “Disciplined Agile for Coaches” book that would include all of the goal diagrams.

A Special Thank You

We’d like to give a shout out to Valentin Tudor Mocanu for answering several of the questions in the chat window while Scott was speaking.  Valentin is both a Certified Disciplined Agile Practitioner and a Certified Disciplined Agile Instructor.

Related Resources

Interested In Having This Presented to You

Please contact the Disciplined Agile Consortium if you would like to have someone present this topic at your organization.

Posted by Scott Ambler on: October 27, 2015 05:25 PM | Permalink | Comments (0)

The Spilled Juice Analogy for Technical Debt

Categories: Technical Debt

Orange Juice Spill

Imagine this: You go home tonight, you walk into your kitchen, and you see that there is orange juice spilled on the floor.  You’re the only one home, you know that you didn’t spill the juice, but there it is on the floor anyway.  This is clearly a problem.  What do you do?

Odds are that you clean it up, because you’re an adult and that’s what adults do.  You don’t walk through the orange juice because that would track it through the rest of your home, making the problem worse.  You could leave the juice there on the floor in the hopes that someone else will come along and clean it up. But if that doesn’t happen you run the risk of your dog lapping it up, and then getting sick from it because citrus is poisonous to dogs.  The problem will have gotten worse. Or perhaps the juice will slowly dry up, attracting all sorts of bugs, once again the problem gets worse.  Additionally, you’ll be forced to constantly walk around the spilled juice thereby slowing you down.  But luckily you’re an adult so you take a few minutes to clean up the juice and thereby avoid these obvious problems.

Now imagine this: You go to work tomorrow, you open up a code file or begin working with a database, and you discover some technical debt.  Perhaps the code isn’t very clear, perhaps the data has inconsistencies, perhaps there isn’t sufficient tests in place, or perhaps there’s a myriad of other issues.  This is clearly a problem.  What do you do?

Odds are that you do nothing about it.  You’d like to do something about it, you yearn to do something about it, but you perceive yourself as not having the authority to do so.  Or perhaps you don’t have the time to fix the problem because you need to focus on getting that new feature out the door.  Or perhaps you don’t have the skills to fix the problem, or the tools to do so, or even recognize that the problem can be fixed (for example, many data quality problems exist because the vast majority of data professionals haven’t learned fundamental techniques such as database testing and database refactoring).  Or perhaps you don’t have the funding to fix the technical debt because the business doesn’t understand it and as a result chooses to fund new development over paying down technical debt.  Or perhaps you see the problem as being overwhelming, not realizing that you can chip away at it a little bit at a time.  Or perhaps you realize that you can safely ignore the problem and let the next person to run into it to worry about it, just like someone in the past has clearly done to you. The point is that you have a litany of excuses to choose from for why you can’t fix the technical debt, so unlike the orange juice spill you don’t clean up the mess.

Unfortunately, just like the orange juice spill, the technical debt problem will only get worse.  You’ll build new code on top of the poor quality code, adding to your overall technical debt.  You’ll propagate the poor quality data through yet another part of your application, adding to your overall technical debt.  You’ll add new tests (hopefully) for the new functionality you’re adding but won’t take the time to add tests in for the existing functionality, adding to the cost of that existing technical debt.  How can this possibly be a good thing for you, or for your organization?

The solution of course is that we need to become the equivalent of software adults, or as many people would say software craftspeople.  This is a choice.  It takes time and it takes investment in yourself.  Just as it took you years to build up the habit of cleaning up spilled juice when you see it (I’m going to go out on a limb and assume that when you were a child you had to be told to clean up the juice many times before you built up the automatic habit) it will also take time to build up the habit of automatically addressing technical debt when you discover it.  While you are on your learning path you’ll also need to convince your colleagues to do the same.  You’ll also need to work with your business stakeholders, the people who fund IT, to understand the implications of technical debt and help convince them to invest in paying it down.  As an organization you need to choose to dig your way out of the technical debt pit.

 

Posted by Scott Ambler on: October 22, 2015 10:01 AM | Permalink | Comments (0)

DevOps Strategies: Enterprise Architecture

DevOps Practices - Enterprise Architecture

The Disciplined Agile (DA) toolkit explicitly includes architecture-related activities, the role of Architecture Owner, and promotes the philosophy of enterprise awareness. Our experience is that agile enterprise architecture proves to be a key enabler for organizations in the process of adopting a Disciplined DevOps mindset.

In addition to general DevOps strategies , there are several enterprise architecture activities that support DevOps:

  • Reuse mindset. An important thing that your enterprise architecture efforts will accomplish is the promotion of a reuse mindset within IT, and throughout your organization in general. Delivery teams with a reuse mindset strive to leverage existing data sources, services, components, frameworks, templates, and many other assets.   This reuse mindset is enabled through education, coaching and mentoring by your enterprise architects (who are ideally active members of IT delivery teams in the role of Architecture Owner).   It is also enabled by technical roadmaps that indicate the technologies that IT delivery teams should, and shouldn’t, be working with. And of course, having high-quality assets that are easy to discover, to understand, and to apply in the course of providing real value to your stakeholders enables reuse.
  • Technical-debt mindset. Your enterprise architecture effort should promote strategies that motivate delivery teams to pay down technical debt when they find it and more important do what they can to avoid it in the first place. Many technical debt strategies are embedded right in DAD, but without a technical-debt mindset this often comes to naught. Enterprise architects, often acting as Architecture Owners on delivery teams, should coach and mentor developers around the issues associated with technical debt. Similarly they should help to educate the senior managers and stakeholders whom they collaborate with in technical debt as well. It requires investment to avoid and remove technical debt, and IT investment decisions are typically in the hands of these people.
  • Development guidelines.  An important aspect of enterprise architecture is the development of guidelines for addressing common concerns across IT delivery teams. Your organization may develop security guidelines, connectivity guidelines, coding standards, and many others. By following common development guidelines your IT delivery teams produce more consistent solutions which in turn makes them easier to operate and support once in production, thereby supporting your DevOps strategy. A potential drawback of common development guidelines is that developers may feel constrained by them. To counteract this problem the guidelines should be developed and evolved in a collaborative manner with the delivery teams, not imposed from above.
  • Technical roadmaps. Your enterprise architecture efforts include the definition, support, and evolution of technical roadmaps that guide the efforts of the rest of the organization (business roadmaps, also important, are the purview of Product Management).  This in turn supports the creation of a common and consistent technical infrastructure within your production environments, enabling common DevOps practices such as continuous deployment, automated end-to-end regression testing, and operational monitoring that we discussed in previous blog postings.

An important aspect of your technical roadmap is to capture both the existing IT infrastructure and the future vision for that infrastructure. Your IT infrastructure potentially includes your network, software services, servers, middleware, and data sources to name a few elements. As you can see in the following diagram, when developing your technical infrastructure vision there are two issues to consider:

  1. Ownership. Does your organization own and operate its own infrastructure or does it outsource some or all of it to external experts?   Outsourcing options include traditional strategies such as having another organization (such as HP or IBM) run your data centers to using cloud-technologies hosted by external organizations (such as Amazon or Google). The advantage of owning your own infrastructure is the greater level of control that it provides you, something that is critical when you must guarantee the security and integrity of your IT solutions. Outsourcing potentially offers greater flexibility in managing your IT infrastructure and cost savings from economies of scale. However, outsourcing requires more sophisticated governance and in the case of traditional strategies is a potential bottleneck when the outsourcer cannot respond in a timely manner to your requests.
  2. Virtualization. Are the elements of your IT infrastructure built to meet the needs of specific solutions or are they softwarized to provide malleability and ease of evolution? With softwarization, also known as software-defined infrastructure (SDI), the elements of your IT infrastructure are fully virtualized. Softwarization includes IT infrastructure models such as a software defined data center (SDDC), software defined storage (SDS), and software defined network (SDN). Softwarization is typically implemented using cloud-based technologies on either side of your firewall. Greater virtualization offers to increase flexibility and programmability of your IT infrastructure, and thereby enabling you to optimize resources.  However, the greater flexibility of virtualization can increase the complexity of your testing efforts and make operational incident simulation more difficult.

IT Infrastructure Strategy Quadrant

 

Related Readings

 

 

Posted by Scott Ambler on: March 17, 2015 02:30 PM | Permalink | Comments (0)
ADVERTISEMENTS

Maybe the dingo ate your baby.

- Elaine Benes

ADVERTISEMENT

Sponsors