Disciplined Agile

Agile development teams build new things every day, and some of these things can be generalized into robust assets for reuse by other teams. This is particularly true for teams that are working with new technologies and techniques: for example, your first few C# teams are likely to develop useful micro services, or the first few agile teams to write personas will develop a potentially reusable template. The downside is that the first people to work with a technique or technology are most likely to make “beginner mistakes,” so their work may not be something you want to share with others. The implication is that you either need to wait to harvest a higher quality asset later or be willing to invest more effort into generalizing the asset.  We’ve found that it’s usually best to wait, giving you more time to gain experience with the technology or technique and discover if there is actually a need for the asset on other teams.

One of the key activities performed by a reuse engineering team is to harvest potentially reusable assets so that they may be generalized for reuse on other teams.  There are five steps to the harvesting process:

1. Find a potentially reusable asset.  Reuse engineers will be constantly on the look out for potentially reusable assets.  They find out about them via word of mouth, through internal online discussions, during enterprise architecture coordination meetings, and from suggestions made by members of development teams.
2. Assess the viability of the asset.  Just because someone thinks that an asset is potentially reusable doesn’t mean that it is.  The reuse engineers must determine whether there is a likely demand for the asset.  When other several other development teams have something similar, or are at least thinking about developing such an asset soon, then this is a fairly straightforward decision.  If there isn’t any immediate customers for the asset then that’s an indication that you are likely better to wait until it’s clear that other teams will be able to leverage the asset before you invest in generalizing it.
3. Generalize the asset. It takes much more effort to generalize an asset to make it reusable than it does to develop it for single-purpose usage. For example, studies showing that development of reusable code costs 111% to 480% of the development cost of single-purpose code.  Generalizing an asset requires great skill because the person doing this needs to be familiar with not only the technology or technique behind the asset but also with how it will potentially be used in practice. Part of generalization is to ensure that the asset conforms to appropriate guidance and making it easy to understand by its intended audience. You also want to validate the asset. You’ll need to unit test code-based assets and review templates and non-code artifacts with their target audiences. Concise overview documentation is important, but far more important are one or two good examples showing how to use the robust artifact properly. In the case of code, your unit tests may be sufficient. For templates, you should capture an actual example of the document for which it is a template (for example, to support a use case template, you should harvest one or two well-written use cases from a project team).
4. Reintegrate the asset into the source. The reuse engineers do this “extra” integration work so that the source team gets the advantage of working with the improved asset without taking on the cost of having to refactor their solution to do so.
5. Publish the asset.  The asset needs to be made available to the teams who could actually reuse it.  The publishing effort typically entails putting the asset into a reuse repository (which could be anything from a folder to a configuration management system to a commercial asset management product) and announcing the availability of the asset to potential customers of it.

There are four basic timing strategies for when to harvest an asset:

1. Harvest in-progress. You generalize an asset developed by a team before it has been released into production.  The reuse engineers do this by working closely with the development team, often joining them for a period of time so that they can generalize the asset and do the work to reintegrate the new version into whatever the team is building.
2. Harvest after production release. With this strategy the reuse engineers wait until the source development team has finished building the asset.   As with the first strategy they do the work to generalize reintegrate the asset.   In cases where the team isn’t working on a new release they do a “patch” release of the solution into production, but this obviously isn’t a desirable situation to be in (it’s far better to have stable development teams).
3. Harvest a legacy asset. In rare situations it is possible to harvest an existing asset currently deployed in production by encapsulating access to it. Although this strategy is often talked about it is rarely used in practice because it requires the asset being harvested to be well architected in that it must be loosely coupled and highly cohesive.  Most legacy assets tend to be the exact opposite, which is the primary reason why they’re not already being reused by other teams.
4. Harvest for a new project. You wait until a project team needs an asset previously developed by another team and then decide to either reuse the asset as-is or to generalize it.  This strategy has the advantage that you know there is a customer for the reusable asset before you invest in it.  However, the team may not be able to wait for the asset to be generalized.  Furthermore, unless the asset was developed recently people are likely to have forgotten about it, implying that this strategy has a short viability time frame.

During the first week of August we ran a mini-survey exploring whether or not teams have fixed iteration lengths.  It was motivated by a question about the results of our Agile Cadences and Technical Debt Survey we ran earlier this year.  Apparently the person was working on a team where their iteration length often shifted and he was wondering how common this was.  I responded that it wasn’t very common and that instead the common practice was to not allow the iteration length to vary.  He found this hard to believe and wondered why we didn’t have any data to back up that claim.  So we decided to run a short survey to discover what is happening in practice.  Over a one-week period we had 264 responses (thank you very much to those of you who did take the time to fill out the survey).

The following diagram summarizes the results of the third question that explored whether teams that were working with iterations kept the iteration length fixed.  As you can see, the majority of agile teams (68%) that have iterations/sprints keep their iteration lengths fixed.  Furthermore, for those teams that allow iteration lengths to vary it’s a rare occurrence (often the result of valid reasons, such as a holiday in the middle of the iteration).

The survey consisted of four questions:

1. The first asked if the respondent was working on an agile team (if not they exited the survey)
2. The second asked how long iterations are (the majority said two weeks or less).  If the team didn’t work with iterations (such as lean teams or continuous delivery teams) or if the respondent didn’t know then they exited the survey.
3. The third question asked whether the length of the iteration was fixed (see the chart above).
4. The fourth question was presented only to people who said that their iteration length could vary.  It explored the reasons why this was the case.  I will summarize these results in a future blog posting.

This posting, the latest in a series focused on a disciplined agile approach to reuse engineering, overviews the activities associated with it. The Disciplined Agile (DA) toolkit promotes an adaptive, context-sensitive strategy.  The framework does this via its goal-driven approach that indicates the process factors you need to consider, a range of techniques or strategies for you to address each process factor, and the advantages and disadvantages of each technique.  In this blog we present the goal diagram for the Reuse Engineering process blade and overview its process factors.

The following process goal diagram overviews the potential activities associated with disciplined agile reuse engineering.

The process factors that you need to consider for reuse engineering are:

1. Obtain assets.  There are several ways that you can obtain potentially reusable assets.  The least effective is to try to build them to be reusable from the very beginning but this strategy often proves problematic in practice because it’s hard to predict what other teams will actually want and as a result you tend to overbuild the asset.  A better approach is to obtain an asset from external sources, either free (as in the case of open source assets) or through purchase.  In this case the assets are often both under and over built – some features you want are missing and many that you do not want are there.  The most effective strategy, in general, is to harvest an existing asset that is already in use within your organization and to generalize it so that others will want to reuse it.
2. Publish assets.  An asset won’t be reused if people don’t know that it exists.  When a new reusable asset is made available it must put into your organizational reuse repository, described appropriately, and announced to any interested parties.
3. Support delivery teams.  There are many ways for a reuse engineering team, if it exists, to support IT delivery teams.  Training, educating, coaching, and mentoring team members in reuse are fairly straightforward to understand.  Some of the more interesting strategies include working with a delivery team to configure and even integrate an asset into their work.  Reuse engineers, often working with a team’s architecture owner, will identify potentially reusable assets that can be harvested and generalized for reuse.
4. Evolve assets.  Reusable assets, like all other assets, will need to be evolved over time.  This includes any work required to generalize the asset to make it reusable, configuration management of the asset’s constituent parts, to update an existing asset to support its evolving purpose, to tailor an asset so that it can be reused in a new situation, and to eventually retire the asset when it is no longer needed.
5. Fund reuse.  There are several ways to fund your reuse engineering efforts.  The least effective, and often debilitating, strategy is to put a chargeback strategy in place.  The basic idea is that if someone reuses an asset then they should pay for it (some organizations will even charge a team for downloading something from their reuse repository, regardless of whether they use it or not).  The problem with this approach is that it in effect punishes teams for reusing things, thereby motivating them to build things from scratch in the future.  In some organizations it proves better to not fund the reuse effort at all, which typically results in ad-hoc reuse at best, than it is to put a chargeback scheme in place.  The most effective approach that we’ve seen is to directly fund the reuse team, thereby taking cost considerations out of the equation when people choose to reuse an asset.
6. Govern reuse. The reuse engineering effort, as with all other efforts, should be governed in a lightweight, collaborative manner.

Related Postings

• Reuse Engineering 101
• What can you reuse on agile development teams?
• Disciplined Agile Reuse: Funding Strategies
• Disciplined Agile Reuse: Harvesting potentially reusable assets
• Disciplined Agile Reuse Engineering Teams (coming soon)
• Reuse Engineering: External Workflow (coming soon)

An important philosophy for succeeding at reuse engineering in the information technology (IT) space is to understand that you have more than one option at your disposal.  You can reuse source code, components, development artifacts, patterns, and templates.   The following diagram summarizes the types, or categories, of reuse available to you.  The left-hand arrow indicates the relative effectiveness of each category – pattern reuse is generally more productive than artifact and framework reuse for example.  Similarly, the right hand arrow indicates the relative difficulty of succeeding at each type of reuse.  Code and template reuse are relatively easy to achieve because you simply need to find the asset and work with it.  Other forms of reuse become hard to achieve; with framework reuse you need to learn the toolkits, with pattern reuse you must learn when to (and when not to) apply various patterns, and with architected reuse you need an effective approach to enterprise architecture in place.

The reuse categories are:

• Architected Reuse.  The identification, development, and support of large-scale, reusable assets via enterprise architecture.  Your enterprise architecture may define reusable domain components, collections of related domain/business classes that work together to support a cohesive set of responsibilities, or service domains which bundle a cohesive collection of services together.
• Pattern Reuse. The use of documented approaches to solving common problems within a specified context.  With pattern reuse you’re not reusing code, instead you are reusing the thinking that goes behind the code.  Patterns can be a multiple levels – analysis, design, and architecture are the most common.  Ward Cunningham’s site is a useful source of patterns on the web.
• Framework Reuse. The use of collections of classes that together implement the basic functionality of a common technical or business domain.  Horizontal frameworks, such as a security framework or user interface frameworks such as the Kendo or QuickUI.   Vertical frameworks, such as a financial services framework, are common in some domains.
• Artifact Reuse. The use of previously created development artifacts – use cases, standards documents, domain-specific models, procedures and guidelines, and even other applications such as a commercial off the shelf (COTS) package – to give you a kick start on a new project.  Sometimes you take the artifact as is and other times you simply use it as an example to give you an idea about how to proceed.
• Component Reuse.  The use of pre-built, fully encapsulated “components” in the development of your solution. In this case a component is self sufficient and encapsulates only one concept.  Component reuse differs from code reuse in that developers don’t have access to the source code.
• Template Reuse. This is the practice of using a common set of layouts for development artifacts such as vision documents, training slide decks, or system overview wiki pages within your organization.
• Code Reuse. The reuse of source code within sections of an application and potentially across multiple applications.  At its best code reuse is accomplished through the sharing of common classes and/or collections of functions and procedures.  At its worst code reuse is accomplished by copying, pasting, and then modifying existing code which adds to your organization’s technical debt and can potentially result in negative overall value.

You can address these reuse categories simultaneously.  Framework reuse often locks you into the architecture of that framework, as well as the standards and guidelines used by the toolkit, but you can still take advantages of the other approaches to reuse in combination with framework reuse.  Artifact and component reuse are the easiest places to start, with a little bit of research you can find reusable items quickly.  However, if your organization doesn’t have a common development process that it follows you may get little benefit from templates.  Pattern reuse is typically the domain of developers with good modeling skills and your enterprise architects should be publishing and providing pattern-oriented guidance to them.

It is important to note that although the diagram indicates that pattern reuse is generally more effective than artifact reuse you may discover that within your organization the opposite holds true.  This may occur because you have a comprehensive collection of reusable artifacts in place, because your organization culture is conducive to artifact reuse, or simply because your developers have little experience with patterns.

Let’s start with some definitions:

• Asset.  An artifact that is retained after its initial purpose is fulfilled.  For example, working source code is an asset because it is retained and potentially updated in the future to address new stakeholder needs.  A user story that is discarded once the functionality it describes is implemented is not considered an asset.
• Robust asset.  An asset that is appropriately documented, generalized beyond the needs of a single team, thoroughly tested, is of high quality, and ideally has several examples to show how to work with it. Robust assets are much more likely to be reused than items without these characteristics.
• Reusable asset.  A robust asset that has been used at least three separate times by at least three separate teams. You can claim that something is reusable, but it isn’t truly reusable until it’s actually been reused; reusability is in the eye of the reuser, not in the eye of the creator.
• Ad-hoc reuse.  An informal approach to reuse where individuals or teams reuse whatever they can find on their own.  Ad-hoc reuse is often a good start.
• Engineered reuse.  A formalized approach to reuse where an organization actively supports a reuse engineering strategy.
• Reuse engineering.   The purposeful creation (or rescue), management, support, and governance of reusable assets.

Why Do We Need Reuse Engineering?

The vast majority of developers, agile or otherwise, take an ad-hoc approach to reuse.  Although this is a good start, we have the opportunity to do much better.  There are several reasons why your organization should consider investing in explicit reuse engineering:

1. Quicker time to market.  The more reusable assets that a team has at its disposal then the less the team will have to build, thereby enabling them to release quicker.
2. Improved return on investment (ROI).  Reuse engineering enables IT delivery teams to avoid building something that your organization already has.  This leads to greater ROI for your IT investment which in turn leads to greater value being delivered to your stakeholders.
3. Improved consistency.  When all of your systems use the same implementation of a service, or component, or function, or framework then that functionality by definition is implemented consistently across those systems.  This makes them more predictable and easier to understand.
4. Easier updates to common functionality.  When functionality is implemented in one place and then reused where needed it is very easy to update that functionality and then deploy the updated version.

Why Reuse Engineering is Difficult

Unfortunately reuse is a lot easier to talk about than it is to implement in practice, at least beyond the ad-hoc level.  There are several reasons why reuse engineering is difficult to achieve:

1. There is a greater impact when reusable assets break.  When an asset is used in only one place and it breaks, then the impact of that breakage is limited to that one place.  When an asset is reused in dozens or hundreds of places, and it breaks, then the impact is significantly larger.  This is reusable assets need to be of high quality.
2. Teams must go beyond the reuse of source code.  Reuse is often described as not “reinventing the wheel,” and an important step for succeeding at reuse is to understand that you have more than one option at your disposal.  For example, in addition to source code you can reuse components, services, patterns, and templates to name a few.  More on this in a future post.
3. Reuse requires enterprise awareness on IT delivery teams.  For reuse to succeed IT delivery teams must understand that reusable assets exist, how these assets fit into your overall IT ecosystem, and what the benefits of reusing them are for your organization.  In Disciplined Agile we promote the philosophy that IT delivery teams should work closely with enterprise architects and reuse engineers, if any exist, so that they will better understand and appreciate these issues.
4. Reuse requires investment.  To get beyond ad-hoc reuse your organization will need to invest in a reuse program.  This may include investment in a reuse engineering team, in a reuse repository, in the development/rescue of potentially reusable assets, and the long-term maintenance and support of those assets.  More on these topics in future postings.
5. Your approach to funding will make or break reuse.  Many organizations will institute chargeback schemes to pay for reuse.  The basic strategy is that just like you would pay for commercial software, you should pay for internally developed or purchased assets within your organization.  That way the cost of those assets is shared fairly across the teams that are actually reusing them.  Unfortunately, in practice, chargeback schemes are guaranteed to kill your reuse engineering efforts because you are in effect punishing teams when they reuse things.  A better strategy is to purposefully fund your reuse engineering efforts and then track, often via automated metrics, the impact those efforts have on your organization (so as to justify the investment).

In future blog postings we will explore a goal-driven approach to reuse engineering as well as the workflows associated with reuse engineering.  It is possible, and highly desirable, for organizations to succeed at reuse engineering.  The Disciplined Agile Reuse Engineering process blade provides the guidance you need to do so.