Disciplined Agile

One of the key aspects of a disciplined agile approach is to be enterprise aware.   The fundamental observation is that your team is only one of many in your organization, except in the case of very small organizations, and as a result should act accordingly.  This means that what your team does should reflect your organization’s overall business and technical roadmaps, that you should strive to leverage as much of the existing infrastructure as possible, that you should try to enhance the existing infrastructure, and that you should work with other teams appropriately so that your overall efforts are complimentary to one another.  This is a straightforward idea conceptually, but in practice acting in an enterprise aware manner can prove more difficult than one would initially think.

Over the years we’ve been asked by several customer organizations to help them to understand how to account for the expense of agile software development.   In particular, incremental delivery of solutions into production or the marketplace seem to be causing confusion with the financial people within these organizations.   The details of accounting rules vary between countries, but the fundamentals are common.  For public companies capital expenses (CapEx) are preferable because they can boost book value through the increase in assets (in this case a software-based solution) and increase in net income (due to lower operating expenses that year).  On the other hand, operational expenses (OpEx) are accounted for in the year that they occur and thereby reduce net income which in turn reduces your organization’s taxes for that year.   Furthermore, in some countries you can even get tax credits for forms of software development that are research and development (R&D) in nature.  In order to get properly account for the expenses incurred by software development teams, and potentially to earn R&D tax credits, you need to keep track of the amount of work performed and the type of work performed to develop a given solution.  Time tracking doesn’t have to be complex: at one customer developers spend less than five minutes a week capturing such information.  The point is that the way that a software developer’s work is accounted for can have a non-trivial impact upon your organization’s financial position.  This in turn implies that the need for agile developers to their track time is a fairly simple example of acting in an enterprise aware manner.

So, I thought I’d run a simple test.  On LinkedIn’s Agile and Lean Software Development group I ran a simple poll to see what people thought about time tracking.  It provided five options to choose from:

• Yes, this is a valuable activity (33% of responses)
• Yes, this is a waste of time (39% of responses)
• No, but we’re thinking about doing so (2% of responses)
• No, we’ve never considered this (18% of responses)
• I don’t understand what you’re asking  (5% of responses, one of which was mine so that I could test the poll)

The poll results reveal that we have a long way to go when it comes to working in an enterprise aware manner.  Of the people inputting their time more of them believed it was a waste of time than understood it to be a valuable activity.  When you stop and think about it, the investment of five minutes a week to track your time could potentially save or even earn your organization many hundreds of dollars.  Looking at it from a dollar per minute point of view, it could be the highest value activity many developers perform that week.

The discussion that ensued regarding the poll was truly interesting.  Although there were several positive postings, and several neutral ones, many more were negative when it came to time tracking.  Some comments that stood out for me included:

• It’s a colossal waste of time unless you’re billing a customer by the hour.
• We record time spent on new development work (as distinct from other tasks such as bug fixing in legacy code and so on) as this is capitalised as an asset and depreciated.
• I think the *most* pointless example was where the managers told us what we should be putting in.
• One day we will move past the “just do it” mentality and have some meaningful conversations and the reasons for what we do.
• In my experience time tracking is a massive waste of time. It’s a poor substitute for management.
• Why do you need to know more than the info available through Sprint Backlog, Sprint burndown and the daily standup?
• Some of my teams (I am SM for three teams) are skeptical about this. They do not think that keeping track of task hours this way will be any more useful than the daily standup reports.  And they do not believe that Management can resist the temptation to use task hours as a measure.

So what can we make of this?  First, it’s clear that delivery teams need a better understanding of the bigger picture, including mundane things such as tax implications of what they’re doing.  Second, it’s also clear that management needs to communicate more effectively regarding why they’re asking people to track their time.  To be fair, management themselves might not be aware of the tax implications themselves so may not be making effective use of the time data they’re asking for.  Third, management needs to govern more effectively.  Several people were clearly concerned about how management was going to use the time data (by definition they are measures) which could be a symptom of both poor communication as well as poor governance (unfortunately many developers have experiences where measures have been used against them, a failure of governance, and no longer trust their management teams to do the right thing as a result).  Fourth, some of the team-focused agile practices, such as burndown charts (or better yet ranged burndown charts) and coordination meetings may be preventing people from become enterprise aware because they believe that all of their management needs are being met by these practices.  Finally, many organizations are potentially leaving money on the table by not being aware of the implications of how to expense software development.

In Summary

Disciplined agilists are enterprise aware.  This is important for two reasons: First, you want to optimize your organizational whole instead of sub-optimize on project-related efforts; second, you can completely miss opportunities to add real value for your organization.  In the anecdote I provided it was clear that some agile developers believe that an activity such as time tracking is a waste, when that clearly doesn’t have to be the case.  Worse yet, although someone brought up the issues around capitalizing software development expenses early in the conversation a group of very smart and very experienced people still missed this easy opportunity to see how they could add value to their organization.  It makes me wonder if some of the agile rhetoric is getting in our way of being more effective as professionals (and, BTW, there are light-weight options for tracking time available to you).

Related Reading

• Strategies for Tracking Time on Agile Teams
• Agile Software Development and CapEx/OpEx

Previously, in Agile Transformation: Being Agile, Doing Agile, and Supporting Agile and Agile Transformation: Comparing Transformation Strategies I discussed the need for your agile transformation efforts to address three factors: People-oriented issues (being agile), process-oriented issues (doing agile), and tooling issues (supporting agile).  I argued that you must focus to a different extent on each of these factors – 80-85%, 10-15%, and 5-10% respectively – and that you need to address all three at once if you’re to successfully transition to agile.  But what is the impetus for becoming more agile in the first place?

The answer is that you want to help people to become more effective so that they can work together to address the success criteria that their stakeholders have set out for them.  The challenge of course is that success criteria varies by team.  Some teams want better time to market, some want better quality, some want improved staff morale, some want improved stakeholder satisfaction with what gets delivered, and some want improved return on investment (ROI) in IT.  Many of course need to deliver on a combination of several of these criteria.

The point is that every team has their own success criteria that they should fulfill.  To do that effectively, agile coaches need to help these teams to “be agile” so that they have the proper mindset and culture to provide a foundation from which they can “do agile”.  To “do agile” teams need to understand, and have the skills to execute, agile practices in such a way that they perform the right practices at the right time to the right extent.  And to do that they need the appropriate tools to support these practices.

Your stakeholders could care less about whether your agile or even about what agile is.  They do care deeply about whether your team is able to meet, and better yet exceed, the criteria set out for them.

In Agile Transformation: Being Agile, Doing Agile, and Supporting Agile we described three factors – people (being agile), process (doing agile), and tools (supporting agile) – that in our experience must be addressed during your agile transformation.  In this blog posting we compare agile transformation strategies in terms of addressing combinations of the three factors.  The following diagram overviews the eight potential strategies that you may embark upon.

Let’s compare each agile transformation strategy one at a time:

1. None of the factors are addressed.  Good luck with that.  ??
2. Focus on people only.  When you focus solely on being agile your teams don’t really learn how agile techniques fit together, nor do they understand the implications of how they’re working.  An example of a people only strategy is to give everyone training and coaching in leadership and collaboration skills and then expect them to self-organize into effective agile teams because “they already have the skills and they’re smart people who can figure it out”.  We saw this sort of strategy fail at a large product company when the teams who received this coaching and training invariably went back to their former ways of working.
3. Focus on process only.  With this approach you get “cargo-cult agile” that is layered on top of your existing processes. What we mean by cargo-cult agile is that the team adopts many of the straightforward management practices, the ones that Scrum tends to focus on, such as holding a daily coordination meeting, an iteration/sprint demo, an iteration/sprint retrospective, and iteration/sprint planning.  You end up with people holding “all of the agile meetings” and who on the surface take on their version of agile roles such as Product Owner and Team Lead.  They have the appearance of working an an agile manner but they really aren’t, and worse yet they don’t even know it and usually nor does senior management.  This is a very common problem when an organization’s entire agile transformation strategy is to send their people on two-day workshops so that they may become “certified masters.”
4. Focus on tools only.  Some organizations believe that if they buy several agile tools, or even an entire agile tool suite, and then force their staff to use them that the tools will somehow make their teams agile.  The actual result is that you end up with a “standardized” approach to agile that is both overly complex (the tools typically include far more features than you’ll ever want, many of which provide functionality completely inappropriate for your situation) and incomplete (you can’t automate everything and even if you could the vendors haven’t gotten to it yet).  The worst offenders are the Application Lifecycle Management (ALM) suites, many of which seem to be offered by vendors who themselves are struggling to deliver consumable products into the marketplace.
5. Focus on people and process. A common agile coaching refrain is to address tooling once you understand the process, which is good advice to an extent, but it quickly falls down when you realize that many technical practices such as continuous integration (CI) and automated testing require adoption of tooling from the very start. When you ignore or at least greatly downplay tooling issues you tend to grow “agile purists”, and even “agile zealots”, who only know how to apply agile in simple contexts.  Ignoring tools can ease your agile transformation at first but eventually tends to get you into trouble when you start to apply agile in more complex situations.
6. Focus on process and tools. When you don’t coach people in the skills and mindset around
   “being agile” you tend to end up with cargo-cult agile with automated bureaucracy.  Agile transformation requires a paradigm shift, which inherently implies you need to address what it means to be agile.
7. Focus on tools and people. When you downplay how to “do agile” you tend to get agile children playing with shiny new toys.  They tend to know the agile language and mindset, and will often have a cursory understanding of simple agile practices that they learned on their two-day certified mastery workshop, but they really don’t know how to successfully build consumable solutions from beginning to end.  Teams who receive little help on “doing agile” tend to spend a lot of time, and a lot of money, figuring out the agile process.  Worse yet, they often adopt a WaterScrumFall approach where Inception and Transition tends to be more traditional and heavy and only Construction is lightweight and agile.
8. Focus on people, process, and tools simultaneously.  When your agile transformation strategy addresses all three factors at once you have the potential to create Disciplined Agile teams that can work at scale.  This only happens when your “being agile” efforts help people to shift to an agile mindset, when your “doing agile” efforts teach a comprehensive yet flexible (think goal driven) view of agile strategies and practices, and your “supporting agile” efforts lead to a context-driven, enterprise aware approach to tool selection.

In your agile transformation you will spend much more effort addressing people-oriented (being agile) issues than you will either of process (doing agile) or tooling (supporting agile) issues.  Think of it like this: these three factors are effectively the legs of a stool, if you don’t address all three then your agile transformation will fall over.

If you’d like help with your agile transformation, please contact us via ScottAmbler.com.

Many organizations struggle to transition to a more agile or lean way of working.  In this blog posting we address three questions:

1. What factors should your agile transformation address?
2. How important are these factors in practice?
3. How difficult are these factors to address?

What factors should your agile transformation address?

Our experience is that successful agile transformations need to address three fundamental issues:

1. People (being agile).  This factor addresses issues such as individual mindset, team and organizational culture, and team and organizational structure.  As a Disciplined Agile coach you must help people to evolve to an agile mindset, learn new skills, adopt improved collaboration strategies, and evolve the way they are organized to reflect their new ways of working.  As you can see below, this factor typically comprises between 80 to 85% of your agile transformation effort.
2. Process (doing agile).  As a Disciplined Agile coach you need to help organizations adopt new agile and lean practices, strategies, and the Disciplined Agile (DA) toolkit itself.  This tends to be between 10 to 15% of the overall transformation effort.
3. Tools (supporting agile).   Agile teams will need to adopt new tools, such as continuous integration (CI) tools, testing tools, and perhaps even agile task board software to name a few.  Agilists will use existing tools, such as their configuration management environment, in new ways.  And they will abandon some existing tools, in particular traditional test management tools, that aren’t applicable in the agile world.  Reworking your tooling strategy tends to take about 5 to 10% of your agile transformation efforts.

How important are these factors in practice?

In the 2014 Agile Transformation survey we asked a series of questions around how important it was to address various people, process, and tooling factors.  The survey respondents had either been through an agile transformation or were currently well into one.  The figure below shows that the first and third most important transformation factors were aligned with being agile, the second and fourth most important factors were around doing agile, and the two least important factors were around tooling (supporting agile).

How difficult are these factors to address?

In the 2014 Agile Transformation survey we asked a similar series of questions around how difficult organizations found it to address various people, process, and tooling factors.  The first and third most difficult factors to address were cultural (being agile).  The second and sixth most difficult factors to address were process oriented (doing agile) in nature.  The fourth and fifth most difficult factors addressed tooling.

Our experience is that if you don’t address all three factors in your agile transformation effort that you will run into serious trouble.  This topic will be explored in our next blog posting.

If you’d like help with your agile transformation, please contact us via ScottAmbler.com.

This posting, the latest in a series focused on a disciplined agile approach to data management, overviews the activities that a disciplined agile data management team may perform. The Disciplined Agile (DA) toolkit promotes an adaptive, context-sensitive strategy to data management.  DAD does this via its goal-driven approach that indicates the process factors you need to consider, a range of techniques or strategies for you to address each process factor, and the advantages and disadvantages of each technique.  In this blog we present the goal diagram for the Data Management process blade and overview its process factors.

The following diagram overviews the potential activities associated with disciplined agile data management.

The process factors that you need to consider for data management are:

1. Improve data quality.  There is a range of strategies that you can adopt to ensure data quality.  The agile community has developed concrete quality techniques – in particular database testing, continuous database integration, and database refactoring – that prove more effective than traditional strategies.  Meta data management (MDM) proves to be fragile in practice as the overhead of collecting and maintaining the meta data proves to be far greater than the benefit of doing so.  Extract transform and load (ETL) strategies are commonplace for data warehouse (DW) efforts, but they are in effect band-aids that do nothing to fix data quality problems at the source.
2. Evolve data assets.  There are several categories of data that prove to be true assets over the long term: Test data that is used to support your testing efforts; Reference data, also called lookup data, that describes relatively static entities such as states/provinces, product categories, or lines of business; Master data that is critical to your business, such as customer or supplier data; Meta data, which is data about data. Traditional data management tends to be reasonably good at this, although can be heavy handed at times and may not have the configuration management discipline that is common within the agile community.
3. Ensure data security.  This is a very important aspect of security in general.  The fundamental issue is to ensure that people get access to only the information that they should and that information is not available to people who shouldn’t have it.  Data security must be addressed at both the virtual and physical levels.
4. Specify data structures.  At the enterprise level your models should be high level – lean thinking is that the more complex something is, the less detailed your models should be to describe it.  This is why it is better to have a high-level conceptual model than a detailed enterprise data model (EDM) in most cases.  Detailed models, such as physical data models (PDMs), are often needed for specific legacy data sources by delivery teams.
5. Refactor legacy data sources. Database refactoring is a key technique for safely improving the quality of your production databases.  Where delivery teams will perform the short term work of implementing the refactoring, there is organizational work to be done to communicate the refactoring, monitor usage of deprecated schema, and eventually remove deprecated schema and any scaffolding required to implement the refactoring.
6. Govern data.  Data, and the activities surrounding it, should be governed within your organization.  Data governance is part of your overall IT governance efforts.

Looking at the diagram above, traditional data management professionals may believe that some activities are missing.  These activities may include:

• Enterprise data architecture.  This is addressed by the Enterprise Architecture process blade.  The DA philosophy is to optimize the whole.  When data architecture (or security architecture, or network architecture, or…) is split out from EA it often tends to be locally optimized and as a result does not fit well with the rest of the architectural vision.
• Operational database administration.  This is addressed by the Operations process blade, once again to optimize the operational whole over locally optimizing the “data part.”

Future blog postings in this series will explore the workflow associated with data management.

Related Resources

• The Data Management process blade
• DevOps Strategies: Data Management
• Disciplined Agile DW/BI: Artifact Creation
• Disciplined Agile DW/BI Teams
• The Enterprise Architecture process blade
• Why Data Management?