Disciplined Agile

Organizations face an increasingly competitive marketplace.  To be competitive, organizations must be able to react to environmental changes and bring new offerings to market quickly.  This requires an adaptive approach to IT, which in turn requires a flexible and malleable approach to enterprise architecture.  To succeed in this new organizational climate enterprise architects require a new mindset, and new ways of working, that reflect the realities that they now face.

In this blog posting we describe a disciplined agile mindset for enterprise architects.  The following terms describe this mindset:

• Collaborative.  First and foremost, disciplined agile enterprise architects (EAs) collaborate closely with their stakeholders.  These stakeholders include business executives, IT executives, IT operations staff, and of course IT solution delivery teams.  Disciplined agile EAs will spend the majority of their time working with stakeholders.
• Learning oriented.  Every day, disciplined agile EAs strive to learn more about the people they are working with, about the business, about the business environment, about the technology and its application, and about the process they are following.
• Sharing. Disciplined agile EAs, and disciplined agilists in general, constantly look for opportunities to share their skills and knowledge with others.  An important part of their job is to help those around them to get better.
• Business focused.  Disciplined agile EAs must have an understanding and appreciation of the business and the way that it operates to effectively interact with, and support, business leaders.
• Multidisciplinary.  Enterprise architects will often start their architecture career as a solution architect, or a data architect, or a business architect, or some other architecture specialty.  Although these are good starting points, enterprise architects need to consider a wider range of issues than those focused on by each of these specialties.  Just as enterprise architecture frameworks such as TOGAF, Zachman, DODAF and others all take a multi-view approach, disciplined agile EAs must have the skills, or be in the process of gaining them, to address a wide range of views.
• Evolutionary. Disciplined agile EAs work in an evolutionary manner.  As you see in the following diagram, when your enterprise architecture team is first formed it may invest a bit of time to perform some initial architecture envisioning.  The team should come to an agreement around their overall vision and get some of the fundamentals captured.  Then the EAs will work with business stakeholders to understand and guide their vision for their lines of business, in particular how they can leverage technology more effectively to provide greater value to the organization.  The EAs will also work closely with IT delivery teams, often taking on the role of architecture owner on the team, so as to help guide and coach the team in architectural skills and concepts. The EAs will also meet on a regular basis, perhaps weekly for a couple of hours, to share what they’ve learned amongst themselves to evolve the architecture assets based on their learnings.  More on this evolutionary approach in a future blog posting.

• Practical.  Disciplined agile EAs are not afraid to get their hands dirty, which is why they will work collaboratively with their stakeholders.  They recognize that for developers, who are important stakeholders of their enterprise architecture efforts, that the proof is in the code.  As a result disciplined agile EAs will produce working code examples as the primary component of their reference architectures.  They realize that developers are much more likely to work with (and then copy) high-quality working code than they are to read a detailed white paper or detailed model extolling the virtues of some architectural concept.
• Pragmatic. Disciplined agile EAs know when to trade-off short and long-term gains and more importantly can help guide their stakeholders through these decisions while coaching them on the implications of what they’re doing.
• Improvement oriented.  Disciplined agile EAs are always looking for ways to streamline the processes that they run into.
• Reuse.  Disciplined agile EAs promote a reuse mindset with both their business stakeholders and with IT delivery teams.  To achieve reuse on a large scale within your organization there needs to be a guiding vision, and that vision is provided by your enterprise architects.  More on this in coming blog postings.
• Technical debt savvy.  Disciplined agile EAs should be a driving force within your organization for addressing technical debt.  They will guide and coach IT delivery teams on both avoiding new debt and on removing old debt.  In parallel they will work with business leaders to understand the implications of technical debt and help guide them in supporting IT delivery teams in addressing it.
• Technical.  And finally yes, disciplined agile EAs must have a technical background so that they understand the implications of the technologies in use (either now or in the future) within their organization.  As noted earlier, a critical strategy for keeping their technical skills fresh is to collaborate with IT delivery teams on a regular basis.

All the features on this list should sound like common sense – that’s because they are.  It’s probable that you’ve worked with enterprise architects in the past whose mindset exhibited many of these features, but did they exhibit all of them?  Likely not.  No matter how good you are, there is always room for improvement.

Related Readings

• The Enterprise Architecture Process Blade
• Agile Enterprise Architecture 101
• Agile Enterprise Architecture Team Structures
• Disciplined Agile Enterprise Architecture: A Goal-Driven Approach
• Disciplined Agile Enterprise Architecture: Workflow
• DevOps Strategies: Enterprise Architecture

This blog posting has been replaced by Disciplined Agile Enterprise Architecture

Is your organization concerned with the cost and time required to adopt agile strategies?  Are you just starting out with agile and hoping to improve your chances of success by learning from the experiences of those who have gone before you?  Are you part way into your agile transformation but struggling to figure out how it all fits together?  If you answered yes to any of these questions please read on.

In this blog posting we discuss what it means to “right size” your software process to meet the needs of the unique situation that your team finds itself in.  We discuss two common anti-patterns: starting with a process framework that is much too large for your needs and starting with one that is far too small for your needs.  We argue that it’s much better to avoid these extremes and instead take a middle-ground approach by starting with a toolkit that is much closer to your actual needs.

Extreme #1: Large process repository

The first process right-sizing anti-pattern is to start with a large process repository, the classic example being IBM’s Rational Unified Process (RUP).  Although RUP is much maligned within the agile community the fact is that if you were to examine it with an open mind that there are many very good ideas promoted by RUP.  Be that as it may. The basic strategy with RUP is that you need to tailor down the process, often dramatically, to meet the unique needs of the situation your team finds itself in.  To do this requires significant process expertise, time, and money.

In practice, however, many organizations ran aground with RUP when they tailored it to be something similar to the waterfall-style processes from yesteryear that they were familiar with.  This is often referred to as RUPifall.  Another common mistake was to say to themselves “wow, there’s a lot of great ideas here, we need to do them all” and as a result they would create a process that was far too heavy to meet their needs.  In either case the problem was that they often didn’t have the process expertise required to right-size their process.  We also see this with organizations starting from frameworks such as ITIL, COBIT, and CMMI so this clearly isn’t just a RUP problem.

Extreme #2: Small methodology

At the other extreme is the right-sizing anti-pattern of starting with a small methodology, the classic example in this case being Scrum.  Although there is significant support for Scrum within the agile community, or at least among people who are new to agile, we’ve been seeing for a long time now that organizations are also running into trouble with this approach too.  With Scrum the idea is that you add in the techniques that Scrum doesn’t address to right-size your process.  Because Scrum proves to be only a very small part of the overall picture, to do this requires significant process expertise, time, and money.

In practice organizations run aground with Scrum because they don’t have the process expertise to expand it to meet their needs.  One only has to count the number of “How does X fit into Scrum?” conversations occurring in agile discussion forums online, or at user group meetings or conferences, to see that this is true.  Step back for a moment and ask yourself how much time and effort has your organization invested in trying to adopt Scrum.  Could it not have been more streamlined?

A few years ago Forrester Research discovered that the majority of organizations “doing Scrum” had actually tailored it into what they called Water-Scrum-Fall (others call this Scrumifall).  As we describe in Going Beyond Scrum this occurs for several reasons.  First, Scrum doesn’t address the full delivery lifecycle, instead choosing to focus on the construction portion of it.  As a result organizations tend to stick with what they know, a heavy project initiation phase and a heavy solution deployment phase.  Second, Scrum only addresses only a small portion of what you need and explicitly leaves technical issues up to you.  These issues include topics such testing, programming, architecture, governance, documentation, deployment and many others.  As a result teams are left to piece together a process strategy that works for them, at the very same time that they’re struggling to understand the fundamentals of agile and lean software development.  They rarely have the process expertise to do that and as a result end up having to hire hordes of expensive Scrum coaches, few of whom seem to understand the enterprise-class realities that your teams actually face.  This is a risky and expensive proposition indeed.

The Effective Middle Ground

Both of these anti-patterns represent extremes: start with something large and cut it down to size, or start with something small and build it up to meet your needs.  Why not start with something much closer to what you actually need?  Doesn’t that make a lot more sense?  Why do you need to do all this process work?  Because someone wants to sell you tooling?  Because someone else wants to sell you expensive coaching and questionable certification strategies?  Isn’t it time to consider a more pragmatic strategy?

The Disciplined Agile (DA) toolkit is a more effective middle ground.  It addresses the full delivery lifecycle, as does RUP (but not Scrum), and even gives you several choices from which to select.  Sometimes a Scrum-based lifecycle is appropriate, sometimes a Lean lifecycle is, other times a continuous delivery lifecycle is best, and sometimes an exploratory “lean start up” lifecycle is.  Different teams, different situations.  DAD starts with a lightweight approach, as does Scrum but not RUP, helping you to avoid the bloatware of RUP and filling in the numerous blanks left by Scrum.  DAD also gives you lightweight tailoring advice in the form of process goal diagrams, in many ways a cross between mind-maps and decision trees, that make your process choices explicit.  The RUP process tailoring advice, if you bother to read it at all, is rather heavy handed and the Scrum tailoring advice boils down to “you’re smart, you can figure it out and if your run into trouble hire a coach.”  Isn’t it time to abandon the extremes?

This middle ground strategy isn’t without its faults.  A challenge with DAD is that it explicitly reveals that agile software development, or as we prefer to say agile solution delivery, is complex.  This is particularly true in enterprise-class situations where teams are often facing combinations of scaling factors such as larger team size, geographic distribution, and regulatory constraints.  DAD makes it explicit that teams need to invest a bit of time up front to perform initial scoping, initial architectural modeling, and initial planning (all in a lightweight manner of course).  This sort of pragmatic thinking can be inconvenient for less-experienced developers who just want to jump in and start coding.  Because DAD promotes the philosophy of enterprise awareness it purposely bakes in strategies for governance, DevOps, and working with IT-level groups such as your enterprise architects and data management team to name a few.  This can also prove to be inconvenient for developers who want to narrowly focus on doing what’s convenient for their team as opposed to what’s best for their organization.

In Summary

The following infographic summarizes the main points in this blog posting.

We hope that you’ve found this blog posting enlightening.  Even if you are well along the way of your Scrum adoption, or of evolving your RUP-based approach, you can still benefit from switching over to DAD.  Scrum teams will find that it addresses many of the issues that you’re still struggling with, and RUP teams will find that it shows how to work in a far more lightweight manner.  Organizations will find that it provides a much better foundation from which to scale agile strategies.

In this continuing series about how the Disciplined Agile (DA) tool kit supports and enhances DevOps, we overview several strategies to help your organization adopt a Disciplined DevOps strategy:

1. Invest in your people.  In our experience 80 to 90% of your overall effort will be in helping people to learn new skills and ways of thinking and to rethink if not abandon many of the “best practices” of yesteryear.  This requires training, education, and coaching over a long period of time – most people will require many months, and sometimes years, to make the transition to this new mindset.
2. Create a safe learning environment.  Teams must be free to experiment, to try new strategies to discover what works for them in the situation that they face.  Very often this will work out well, with a few stumbles along the way, but every so often the experiment will show that the strategy just isn’t right for this team.  That should still be considered a successful experiment, learning what doesn’t work is just as valuable as learning what does, and the team should not be worried about recrimination for “failed” experiments.
3. Look at the “whole system”.  Disciplined DevOps is about more than just continuous delivery (although that is a great start) and in most enterprises it’s about more than just streamlining development and operations.  With a Disciplined DevOps mindset we strive to improve flow through the entire ecosystem, including development, operations, support, enterprise architecture, data management, release management, and most importantly to the business itself.
4. Improve locally, transform globally.  Each team, including your solution delivery teams, your enterprise architecture team, your operations staff, and many others must strive to improve and streamline the way that they work.  These local improvement efforts must be supported by a “global” transformation effort that focuses on improving DevOps across your entire ecosystem.  Every team will affect other teams, motivating them to make improvements which in turn affects how they work with others.  Your IT department is a complex adaptive system where people and teams learn and improve over time in a dynamic, evolutionary manner.  If you just focus on local improvements your DevOps effort is likely to devolve into chaos.  If you just focus on a company-wide, global transformation it is likely to get bogged down in bureaucracy.  An “improve locally, transform globally” approach is a viable middle ground that benefits from these two extremes while avoiding the disadvantages.
5. Have a communication plan (and work it).  Adopting a Disciplined DevOps strategy within your organization typically requires a lot of (often small) changes.  Although it may be clear to you why this is important it isn’t always clear to everyone else.  People need to understand why you’re making these changes, what’s in it for them, what the overall change strategy is, how far along the plan you currently are, what changes are coming soon, and so on.  You communication plan may include regular newsletters, posters overviewing key concepts, brown bag lunches where people share their experiences, electronic discussion forums, management presentations, and many more.  The keys to success are to have a constant drum beat of information, to be as open and honest about what is actually occurring, to provide opportunities to everyone to learn, and to motivate everyone to share their learnings.
6. Think long-term.  Disciplined DevOps is a journey, not a destination.  It takes time for people to adopt a new mindset, months and often years before it is truly ingrained in their way of thinking.  This paradigm shift does not occur by management fiat, nor does it occur as the result of a day or two of training (although training is important), nor does it occur because you’ve invested in new tooling.  Organizations that successfully make this paradigm shift do so by investing in their people, their process, and their tooling over the long term.

An important financial governance concern in modern enterprises is how to expense costs properly.  This of course includes the costs of IT, including the costs associated with agile software development teams.  There is more to this of course than just adding up expenses, these expenses need to be categorized appropriately and in some cases can have a measurable impact on your bottom line.  In this blog posting we explore how to categorize agile software development costs into either capital expenses (CapEx) or operational expenses (OpEx).

Some Basics

Let’s begin with a few important observations:

1. How you expense cost (and revenue for that matter) is driven first by accounting rules, such as Financial Accounting Standards Board (FASB) in the US, Accounting Standards for Private Enterprises (ASPE) in Canada, and International Financial Reporting Standards (IFRS) IAS 38 in general.  The implication is that CapEx/OpEx decisions in part are driven by geography.  At the end of this posting is an appendix summarizing these regulations.
2. There are a few categories of expense, such as training and education (including coaching related expenses), which automatically fall into the operational expense category regardless of when they occur in the lifecycle.
3. Where the appropriate accounting rules provide leeway, for example financial minimums for what to consider for capitalization, your corporate policy will need to guide you.  A common problem in established organizations is that some of this guidance was set before agile approaches were adopted and reflect waterfall-style governance.
4. How you account for expenses is orthogonal to development paradigm.  It doesn’t matter what method the team follows, the actual capitalization rules remain the same.  The implication is that you need to determine how to apply these rules effectively for each paradigm, and as you’ll see even by lifecycle.
5. This decision, or as we’ll soon show reporting activity, is typically the purview of Finance, not IT.  You really shouldn’t have to worry too much about this.
6. You must engage the appropriate Finance, IT, and Audit people to ensure that everyone understands the implications of how you’re working and agrees to how you will approach accounting for expenses.

Why is This Important?

For publicly traded companies CapEx can potentially boost book value through the increase in assets (in this case a software-based solution) and increase in net income (due to lower operating expenses that year). On the other hand, OpEX is accounted for in the year that it occurs and thereby reduces net income which in turn reduces your organization’s taxes for that year.

How DAD Helps

As you can see in the appendix most of the rules for when to capitalize boil down to “start capitalizing when you know its real”, something that works well with DAD’s risk-based milestones.

As you can see in the diagram, we’ve identified two potential points where you can start capitalizing software development costs:

• Conservative: Stakeholder vision milestone.  This is the point which most organizations use as their CapEx starting point.  As you can see in the appendix, this milestone lines up perfectly with the regulations – you’ve made the decision to go forward with development, you’ve got a technical architecture strategy in place, you have an understanding of what you hope to produce, and so on.
• Aggressive: Project/product team start. Some organizations will choose to start capitalizing at the very beginning of a software development project because they have effectively made the decision to move forward with the effort at that point, they know what the architecture is (due to a good understanding of their existing environment and technical road map), and due to their culture it is highly unlikely that they will cancel the effort.

Of course, there are a few more issues to consider:

• You cancelled the project.  If you cancel a project (or release of a product) most organizations will choose to reverse their capitalization decision for any expenses incurred with that project/release.
• You’re prototyping.  Prototyping, including time spent on proof-of-concept (PoC) or spikes, is usually considered to be an operational expense because it is a learning activity.  This is the case regardless of when in the lifecycle this work occurs.
• Your team follows the continuous delivery lifecycle.  Teams following this lifecycle are in the Construction and Transition efforts all the time, so they’re effectively past the Stakeholder Vision milestone (the Conservative point) at all times.
• Your team follows the exploratory/lean start up lifecycle.  Teams following this lifecycle are not yet at the point where they’ve made the decision to continue with development, and when they do so the typically kick into one of the other lifecycles anyway.  The implication, at least from a conservative point of view, is that these expenses should be operationalized.

Time Tracking

Needless to say, many developers, particularly agile ones, feel that time tracking is a waste.  Spending 5 minutes a week is ok, and to be quite blunt should be more than sufficient, but spending fifteen minutes or more a day doing so is far too much.  For CapEx/OpEx purposes you merely need to track broad categories of software development work such as Development, Prototyping, Vacation, and Training/Education.  These categories will be determined by the applicable accounting rules and should be kept to a minimum.

Our advice is to keep your time tracking strategy as simple as possible, make it very clear to everyone why it’s important to capture their time accurately, and do not use the data to directly measure individuals.  You may find the article Why Time Tracking Might Be the Most Valuable Activity an Agile Developer Performs to be an interesting read.  And, as you would expect from Disciplined Agile, we compare and contrast several Strategies for Tracking Time on Agile Teams.

It’s important to note that tracking time simply for CapEx/OpEx reasons is very hard to justify in practice due to the overhead that it injects plus the loss in team morale (nobody likes tracking time).  Another option is to simply track the amount of time that the entire team spends in a week and then assign a percentage of that time to each category.  This would require a slight amount of effort to track, likely something done by your Team Lead.

The Bottom Line

CAPEX/OPEX decision is fairly straightforward for teams that are following a Disciplined Agile Delivery (DAD) approach.  The team simply needs to keep track of important milestone dates (either the Project/Team Start date or the Stakeholder Vision acceptance date) and track their time in a few broad categories as described earlier.  If you do that, it’s a simple matter of running a report against your time data.

Appendix: What The Regulations Say

Disclaimer: You must read the actual regulations for yourself, what follows is just a summary and may become out of date overtime as the regulations evolve.

US FASB 

• Internal and external costs incurred during the preliminary project stage shall be expensed as they are incurred
• Internal and external costs incurred to develop internal-use computer software during the application development stage shall be capitalized
• Costs to develop or obtain software that allows for access to or conversion of old data by new systems shall also be capitalized
• Training costs are not internal-use software development costs and, if incurred during this stage, shall be expensed as incurred
• Data conversion costs, except as noted above, shall be expensed as incurred
• Internal and external training costs and maintenance costs during the post implementation-operation stage shall be expensed as incurred

International Financial Reporting Standards (IFRS) – IAS 38

You must be able to demonstrate ALL of following BEFORE you can START capitalizing costs:

• Technical feasibility of completing the intangible asset so that it will be available for use or sale
• Its intention to complete the intangible asset and use or sell it
• Its ability to use or sell the intangible asset
• How the intangible asset will generate probable future economic benefits. Among other things, the entity can demonstrate the existence of a market for the output of the intangible asset or the intangible asset itself or, if it is to be used internally, the usefulness of the intangible asset

Note: It is expected that FASB will recognise IAS 38 in 2016.

Canada – Accounting Standards for Private Enterprises (ASPE)

Based on ASPE, you must be able to demonstrate ALL of following BEFORE you can START capitalizing costs:

• Technical feasibility of completing the intangible asset so it will be available for use / sale
• Intention to complete the intangible asset and use / sell it
• Ability to use / sell the intangible asset
• Availability of adequate technical, financial and other resources to complete development and use / sell the intangible asset
• Ability to reliably measure the expenditure related to the intangible asset during development
• How the intangible asset will generate probable future economic benefits

Related Reading

• The Secure Funding process goal
• The Finance process blade