Disciplined Agile

An important, yet seldom discussed, topic is how do you govern agile software development teams?  This is rather strange considering that agile teams are in fact being governed whether you choose to recognize this or not.  If someone is keeping an eye on the budget, or the level of quality being produced, or whether you are building something of value for your stakeholders then you are being governed.  If there are defined roles and responsibilities for team members then you’re being governed.  If you are following common programming or database guidelines, or working towards a common technical or business roadmap, or inviting outsiders to attend your coordination meetings then you are being governed.  In this blog posting we explore what is, and what isn’t, agile delivery governance

What Is Agile Delivery Governance?

Governance establishes chains of responsibil­ity, authority and communication in support of the overall enterprise’s goals and strategy. It also establishes measurements, policies, standards and control mechanisms to enable people to carry out their roles and responsibilities effectively. You do this by balancing risk versus return on investment (ROI), setting in place effective processes and practices, defining the direction and goals for a team, and defining the roles that people play within a team.

Agile delivery governance is the governance of agile delivery teams in a manner that reflects and supports the agile paradigm.  We have identified the following principles for effective agile delivery governance:

1. Collaboration with delivery teams is more effective than trying to force them to conform.  IT professionals are intellectual workers, the type of people whom are more likely to do what you want when you work with them to do so rather than tell them to do so.  For example, if you want an agile delivery team to work towards a common technical strategy (perhaps captured by your organization’s technical roadmap) you are better off having people knowledgeable with that strategy to work directly with the team rather than forcing them to fill out specific documents that will be reviewed at some point.  In this case the toolkit suggests the specific role of Architecture Owner who is responsible for this.
2. Enabling teams to do the “right thing” is more effective than trying to inspect it in. Agile team members are human, and being human their natural tendency is to do the easiest thing possible.  The implication is that for things that you want to have happen you should enable the teams to do those things, to make them easy to do.  For example, say your organization wants developers to follow common coding conventions.  One way to do this would be to hold code inspections, a time-consuming process, that validates whether the coding conventions are being followed.  An easier approach would be to adopt a code analysis tool such as CheckStyle and include it in your continuous integration (CI) strategy.  This automated approach requires no extra effort on the part of the developer and provides immediate feedback as to the quality of their code, providing them with a learning opportunity to help them improve their skills.
3. Continuous monitoring provides more timely insight than quality gate reviews.  Team dashboards that use business intelligence (BI) technology to display real-time measures generated by the use of your development tools have become very common in the past few years.  This enables both the team and their stakeholders to monitor the team’s progress in a continuous real-time manner.  This is orders of magnitude more effective than traditional “quality gate” reviews of artifacts because the information displayed on the dashboards is automatically generated as a side effect of tool usage and therefore incredibly difficult to fake, whereas team artifacts (status reports, specifications, plans, and so on) are hand-crafted and thus contain whatever information the creator(s) decide to capture.  Furthermore, after the initial cost of the dashboard technology this approach is effectively free.  In the original DAD book we referred to this strategy as development intelligence (DI).
4. Transparency into teams is provides better insight than status reports.  Through application of strategies such as information radiators, team dashboards and active stakeholder participation, the work of a disciplined agile delivery team is effectively transparent.  All of these strategies are described below.  As a result your stakeholders can discover what the current status of your team is simply by looking whenever they want, they don’t need to rely on status reports crafted by the team that may be little more than works of fiction.

What Isn’t Agile Delivery Governance?

Governance often has a bad name with developers, typically because they have had ineffective traditional strategies inflicted upon them for years.  These traditional strategies often prove to be little more than a layer of additional bureaucracy that offers little or no value to your organization.  To put your mind at ease on this issue, we’d like to point out that agile delivery governance isn’t:

• Documentation driven.  The greater levels of collaboration promoted by agile, in combination with the strategy of team dashboards alleviates the need for many of the artifacts that traditional governance strategies dictate.  Furthermore, as you will soon see, the milestones suggested by the DA toolkit are risk-driven, not documentation driven.
• Onerous.  An onerous governance strategy will not be followed by delivery teams, at best they will do just enough work to make it appear that they are following the governance process, which implies that the process has utterly failed.  As a result we’ve kept the governance strategies as light-weight as we possibly can, and when you do it right effective governance actually improves the productivity of delivery teams instead of detracting from it.  These light-weight strategies are described below.
• Management.  Governance and management are two different things. Governance looks at a team from the outside, treating it as a system that needs to have the appropriate structure and processes in place to provide a stream of value. Management, on the other hand, occurs inside the team and ensures that the structure and processes are implemented effectively.
• Optional.  Your teams are being governed, like it or not. Our philosophy is that you deserve to be governed well, which is why we’ve taken the time to describe how to do so.

You are being governed.  We believe that you deserve to be governed well.  For more on this topic, we suggest reading the article Governing Agile Teams.

This posting, the latest in our series focused on a disciplined agile approach to people management, overviews the activities associated with it. The Disciplined Agile (DA) toolkit promotes an adaptive, context-sensitive strategy.  DA does this via its goal-driven approach that indicates the process factors you need to consider, a range of techniques or strategies for you to address each process factor, and the advantages and disadvantages of each technique.  In this blog posting we present the goal diagram for the People Management process blade and overview its process factors.

The following process goal diagram overviews the potential activities associated with disciplined agile people management. These activities are performed by, or at least supported by, your people management (often called a human resource) team.

The process factors that you need to consider for people management are:

1. Manage staff.  Your organization needs to perform basic functions such as hiring (onboarding) staff, letting people go (offboarding), promoting, demoting, transferring them and providing benefits to people.
2. Organize IT.  What is your strategy for organizing your IT department?  Do you do it by job function (e.g. have a business analyst group, a project management group, and so on), by geography (e.g. a North American IT department, a European IT department, and so on), by business division (e.g. an IT group to support Retail banking, an IT group to support brokerage, and so on), or by value creation (e.g. an IT group to support a specific product line).  Or combinations thereof?
3. Guide careers.  Your organization should support the career aspirations of its staff, providing opportunities to people and supporting their efforts to achieve their goals.
4. Staff IT.  You need to identify, and plan for, your organization’s staffing needs.  This includes succession planning for senior IT people, critical technical positions (yes, that includes all those legacy COBOL programmer positions), and other critical roles such as product owners.  This also includes staff capacity planning/forecasting as well as determining your mix of full time employees (FTEs) and contractors.
5. Reward staff.  There are many ways that people and teams can be rewarded, including base pay, bonuses, and non-monetary rewards.  For some people in some organizations their pay is publicly known (for example, in Canada public employees who make over a certain amount have their salaries published annually) whereas for most people their remuneration strategy is private.
6. Form teams.  There are different types of teams that can be formed to address IT functions, each of which are (self) organized differently.
7. Evolve teams.  Team membership and structure evolve over time, and there are several common strategies that enable this.  Some teams are ad-hoc, forming when their needed and disbanding when they’re not, with little or no management intervention.  Sometimes people are assigned to teams and sometimes people volunteer to be on a team.  Some organizations are adhocracies where teams are self-organizing and have defined strategies for enabling collaboration and communication between teams.
8. Govern people management.  Your people management activities, just like all other activities, should be governed effectively.  An important aspect of people management governance is the definition of roles and responsibilities (see Roles on DAD Teams and DA Roles at Scale for suggestions), as is the usual measurement and monitoring activities.

For more details, please read the article People Management.

People management goes by many names, including human resource (HR) management, talent management, staff management, and work force management to name a few.  The fundamental goal of people management is to attract and retain great people who work on awesome teams.

There are several reasons why people management is important for your IT organization:

1. People and the way they work together are your primary determinant of success. Software-based solutions are built by a team of people working together, and IT organizations are a collection of teams working together to support the rest of your enterprise. The implication is that you need to attract and retain the right people and build awesome teams comprised of these people.
2. You want to support people’s career aspirations. To retain top talent your organization needs to help these people remain so by providing opportunities for fulfilling work, training and coaching in new skills and new ways of thinking, and in mentoring.
3. Greater employment flexibility attracts a wider range of people.  To what extent will your organization support flexible working hours, flexible working locations (e.g. allowing people to work from home), flexible device options (e.g. BYOD), job sharing strategies, and many more strategies?  Greater flexibility increases the attractiveness of your organization at the cost of requiring more robust collaboration, management and governance strategies.  One employment strategy does not fit all.
4. Many people-oriented activities fall outside the scope of what occurs on your work team(s). The hiring of people, people leaving the company, moving between teams, getting trained in skills not directly related to their current team efforts, and many more activities partially or fully land outside the scope of an IT delivery team. Yes, a team should be actively involved in the decisions surrounding who is on the team but that doesn’t imply that they do all of the work surrounding the hiring process.
5. Legal requirements.  Every organization must conform to the laws of the territories in which they operate, and there are always laws around how organizations can treat the people that work for them.  These laws vary by country, and sometimes even by territories within countries, and evolve constantly.  The laws pertaining to how you hire, reward, and fire someone in San Francisco are different than the laws for someone in Toronto which are different again than the laws for someone in Moscow.
6. Organizational sustainment.  Your organization has long-term staffing needs, including succession and capacity planning. Succession planning focuses on identifying and supporting the people who are being groomed to fill key positions in the future. Capacity planning focuses on ensuring you will have enough people with the right skills in the right places at the right times to get the work done in the future.
7. You need to manage your staffing mix.   There are several employment options available to people: They may be full-time employees (FTEs) of your organization, they may be independent contractors working for a defined period of time with your organization, employees of external service providers who are assigned to work on your teams, or they may be consultants working with your organization on more of as-needed, ad-hoc basis.  Each of these employment options have advantages and disadvantages and your organization needs to actively manage their overall staffing portfolio to ensure that they are meeting their long-term needs.  This is an aspect of capacity planning.

For more details, please read the article People Management.

On Wednesday October 21, 2015 we hosted a webcast entitled “Crushed by Technical Debt?” A PDF of the slides can be found on Slideshare.  The webcast addressed the following issues:

• What is technical debt?
• How can we avoid technical debt?
• How do we identify technical debt?
• How do we fix technical debt?
• When should we accept technical debt?
• How can we fund technical debt?

As you can imagine we ran out of time to answer all of the questions that we received during the presentation. Below are answers to all of the questions we received, even the ones that were answered during the Q&A portion of the webcast. The questions, and their answers:

• Stories for technical debt?
• How to convince product owners about technical debt?
• Too many warnings from your code analysis tools?
• Too much technical debt?
• Measuring technical debt?
• Fix or start from scratch?
• A negative awareness score for senior business management?
• Forecasting technical debt?
• Can architecture owners work on multiple teams?
• Where are the goal diagrams?

Additional topics:

• A special thank you
• Related reading

Question: What’s your opinion regarding separating tech debt into separate stories?

This is one way of doing it. At one point during the presentation we discussed three strategies for how to fund paying down technical debt.  These three strategies are:

1. Technical debt removal as regular investment. The idea is that developers automatically pay down technical debt, just like you would automatically clean up spilled juice in the middle of your kitchen floor, whenever you run into it.  In effect paying down technical debt becomes part of your culture.
2. Stories to address technical debt. The idea is that you have requirement artifacts, such as technical stories, to remove a small amount of technical debt within your environment.
3. Projects to address technical debt. With this approach you have specific projects to rewrite portions of an IT solution, or perhaps even to remove that solution completely.

Question: How to convince Product Owners to address Technical Debt?

Product Owners (POs) are responsible for prioritizing the work. When technical debt is addressed via stories then the PO can easily deprioritize such stories.  The implication is that the team members, and in particular the Architecture Owner, must work closely with the PO to help them to understand the implications of technical debt and why it’s important to pay it down.

When technical debt is addressed automatically via regular investment, it is a built in overhead that affects your team’s overall velocity (negatively in the short term, but positively in the long term.

Question: What do you do when there are to many warnings thrown by your code analysis tools?

Regarding using code-analysis output as a metric: do you have any specific advice for dealing with the scenario with legacy code where it will initially overwhelm you with thousands of issues, from the trivial (i.e. spelling mistakes) to major : i.e. should one start by switching off the trivial warnings and deal with the major, or start fixing the big-numeric-count issues no matter how trivial just to get the numbers down?

It’s harsh to say, but the reason why you’re getting a lot of warnings is because the code sucks. When this is the case, it’s important to recognize the situation for what it is and get going on fixing the problems.

Initially I would turn off a lot of the lower priority checks so that the team could focus on the critical issues with their code. Once they refactor those away then I would turn on the next group of high-priority checks and so on until the problems are dealt with.  This could take a long time, so you’ll need the discipline to stay at it.  Perhaps start each retrospective with the question “How many code checks can we turn on this coming iteration?”

Question: Lack of (enough) clean code from the first time?

This is always a major frustration for people.   It is incredibly rare to be in a situation where clean code was written from the very beginning and then kept clean via techniques such as refactoring.   Instead most people find themselves swimming in legacy code, data sources, user interfaces, and so on that contain lots of technical debt.  Our advice is to accept:

• The situation that you find yourself in
• That the technical debt isn’t going to fix itself
• It’s only going to get worse
• You need to start paying down your technical debt now

If it makes you feel any better, and it likely won’t, the 2014 Software Development at Scale survey found that 92% of agile teams were facing some form of technical complexity.

Question: How we should measure technical debt?

There are many different ways to measure technical debt, all focused around solution quality. You may choose to measure code quality, perhaps in an automated fashion via code analysis tools, for example.  You may also choose to track defect trends over time, although that can easily become a measure of how good you are at testing instead of the quality of what you’re producing.

Question: How do I know if it’s better to fix technical debt or start all over from scratch?

This typically comes down to a gut feel based on your experience doing such work in the past.

Question: What does a negative score for technical debt awareness means? How should that be read & interpreted?

This is a reference to one of the charts from the 2015 Agile Cadences and Technical Debt Survey, see below , where senior business management had a negative score.  The range is from -10 (very unaware) to +10 (very aware).  So on average business management leaned towards being unaware of the implications of technical debt.  This is a problem seeing as the business needs to fund the removal of technical debt.

Question: Is there any best practice/method to forecast Technical Debt?

Unless you really have your act together, the forecast is that technical debt is going to get worse over time.  Seriously though, that’s why you need to have a technical debt metrics program in place. In this case you want to track the trends over time, and simply extrapolate those trends to forecast the implications to your organization of technical debt.

Question: Can one architect span multiple teams? That is, is it a good idea to have the same person as the architect across multiple teams?

In Disciplined Agile we explicitly include the role of Architecture Owner (AO), which was adopted from the Agile Modeling methodology. This is effectively an agile solution/application architect who works in a collaborative, evolutionary, and facilitative manner.  Every delivery team should have someone in the AO role to help guide them through architectural issues.  Ideally each team has one person in this role and they are dedicated to the team full time, just like other team members.

Having said that, as we like to say “you go to war with the army that you’ve got.” For example, you may have ten agile teams but currently only four people with the skills to be an AO.  In this situation each AO is going to be a member of two or three teams.  As a result they will suffer a personal productivity loss from task switching and their teams will suffer a productivity loss from having to sometimes wait for them to be available.  This isn’t ideal but it’s the reality that many organizations face.  The solution is for the AOs to work collaboratively with other team members so as to start passing on their skills and knowledge to them, to help grow their fellow team members.

Question: Are the goal diagrams available in a single document?

Great question. Sadly the quick answer is that there isn’t a print-based one at the current time.  Our focus has been on publishing to the web, ignoring the fact that we recently published the 104-page book Introduction to Disciplined Agile Delivery. We have published a landing page for the Process Goal Diagrams for Disciplined Agile Delivery. We are also actively publishing the goal diagrams for the process blades, such as Enterprise Architecture and Portfolio Management, as well.  The easiest way to access them is from the diagram on the Disciplined Agile Home Page (just click on the corresponding bubbles). This is a work in progress and we hope to have the process blades descriptions in place by December.

Having said that, we have been thinking about publishing some sort of “Disciplined Agile for Coaches” book that would include all of the goal diagrams.

A Special Thank You

We’d like to give a shout out to Valentin Tudor Mocanu for answering several of the questions in the chat window while Scott was speaking.  

Related Resources

• Article: 11 Strategies for Dealing with Technical Debt
• Slides (PDF): Crushed by Technical Debt?

Imagine this: You go home tonight, you walk into your kitchen, and you see that there is orange juice spilled on the floor.  You’re the only one home, you know that you didn’t spill the juice, but there it is on the floor anyway.  This is clearly a problem.  What do you do?

Odds are that you clean it up, because you’re an adult and that’s what adults do.  You don’t walk through the orange juice because that would track it through the rest of your home, making the problem worse.  You could leave the juice there on the floor in the hopes that someone else will come along and clean it up. But if that doesn’t happen you run the risk of your dog lapping it up, and then getting sick from it because citrus is poisonous to dogs.  The problem will have gotten worse. Or perhaps the juice will slowly dry up, attracting all sorts of bugs, once again the problem gets worse.  Additionally, you’ll be forced to constantly walk around the spilled juice thereby slowing you down.  But luckily you’re an adult so you take a few minutes to clean up the juice and thereby avoid these obvious problems.

Now imagine this: You go to work tomorrow, you open up a Word document or begin working with a data from an existing database, and you discover some technical debt.  Perhaps the wording in the document isn’t very clear or perhaps the data has inconsistencies, or there’s a myriad of other issues.  This is clearly a problem.  What do you do?

Odds are that you do nothing about it.  You’d like to do something about it, you yearn to do something about it, but you perceive yourself as not having the authority to do so.  Or perhaps you don’t have the time to fix the problem because you need to focus on getting that document to your boss.  Or perhaps you don’t have the skills to fix the problem, or the tools to do so, or even recognize that the problem can be fixed (for example, many data quality problems exist because the vast majority of data professionals haven’t learned fundamental techniques such as database testing and database refactoring).  Or perhaps you don’t have the funding to fix the technical debt because your leadership doesn’t understand it and as a result chooses to fund new work over paying down technical debt.  Or perhaps you see the problem as being overwhelming, not realizing that you can chip away at it a little bit at a time.  Or perhaps you realize that you can safely ignore the problem and let the next person to run into it to worry about it, just like someone in the past has clearly done to you. The point is that you have a litany of excuses to choose from for why you can’t fix the technical debt, so unlike the orange juice spill you don’t clean up the mess.

Unfortunately, just like the orange juice spill, the technical debt problem will only get worse.  You’ll build write new information in addition to the poor quality writing, adding to your overall technical debt.  You’ll propagate the poor quality data through yet another part of your application, adding to your overall technical debt.  You’ll add new tests (hopefully) for the new functionality you’re adding but won’t take the time to add tests in for the existing functionality, adding to the cost of that existing technical debt.  How can this possibly be a good thing for you, or for your organization?

The solution of course is that we need to take responsibility.  This is a choice.  It takes time and it takes investment in yourself.  Just as it took you years to build up the habit of cleaning up spilled juice when you see it (I’m going to go out on a limb and assume that when you were a child you had to be told to clean up the juice many times before you built up the automatic habit) it will also take time to build up the habit of automatically addressing technical debt when you discover it.  While you are on your learning path you’ll also need to convince your colleagues to do the same.  You’ll also need to work with others to understand the implications of technical debt and help convince them to invest in paying it down.  As an organization you need to choose to dig your way out of the technical debt pit.

Related Reading

• 11 Strategies for Dealing With Technical Debt