Disciplined Agile

Previously, in Agile Transformation: Being Agile, Doing Agile, and Supporting Agile and Agile Transformation: Comparing Transformation Strategies I discussed the need for your agile transformation efforts to address three factors: People-oriented issues (being agile), process-oriented issues (doing agile), and tooling issues (supporting agile).  I argued that you must focus to a different extent on each of these factors – 80-85%, 10-15%, and 5-10% respectively – and that you need to address all three at once if you’re to successfully transition to agile.  But what is the impetus for becoming more agile in the first place?

The answer is that you want to help people to become more effective so that they can work together to address the success criteria that their stakeholders have set out for them.  The challenge of course is that success criteria varies by team.  Some teams want better time to market, some want better quality, some want improved staff morale, some want improved stakeholder satisfaction with what gets delivered, and some want improved return on investment (ROI) in IT.  Many of course need to deliver on a combination of several of these criteria.

The point is that every team has their own success criteria that they should fulfill.  To do that effectively, agile coaches need to help these teams to “be agile” so that they have the proper mindset and culture to provide a foundation from which they can “do agile”.  To “do agile” teams need to understand, and have the skills to execute, agile practices in such a way that they perform the right practices at the right time to the right extent.  And to do that they need the appropriate tools to support these practices.

Your stakeholders could care less about whether your agile or even about what agile is.  They do care deeply about whether your team is able to meet, and better yet exceed, the criteria set out for them.

In Agile Transformation: Being Agile, Doing Agile, and Supporting Agile we described three factors – people (being agile), process (doing agile), and tools (supporting agile) – that in our experience must be addressed during your agile transformation.  In this blog posting we compare agile transformation strategies in terms of addressing combinations of the three factors.  The following diagram overviews the eight potential strategies that you may embark upon.

Let’s compare each agile transformation strategy one at a time:

1. None of the factors are addressed.  Good luck with that.  ??
2. Focus on people only.  When you focus solely on being agile your teams don’t really learn how agile techniques fit together, nor do they understand the implications of how they’re working.  An example of a people only strategy is to give everyone training and coaching in leadership and collaboration skills and then expect them to self-organize into effective agile teams because “they already have the skills and they’re smart people who can figure it out”.  We saw this sort of strategy fail at a large product company when the teams who received this coaching and training invariably went back to their former ways of working.
3. Focus on process only.  With this approach you get “cargo-cult agile” that is layered on top of your existing processes. What we mean by cargo-cult agile is that the team adopts many of the straightforward management practices, the ones that Scrum tends to focus on, such as holding a daily coordination meeting, an iteration/sprint demo, an iteration/sprint retrospective, and iteration/sprint planning.  You end up with people holding “all of the agile meetings” and who on the surface take on their version of agile roles such as Product Owner and Team Lead.  They have the appearance of working an an agile manner but they really aren’t, and worse yet they don’t even know it and usually nor does senior management.  This is a very common problem when an organization’s entire agile transformation strategy is to send their people on two-day workshops so that they may become “certified masters.”
4. Focus on tools only.  Some organizations believe that if they buy several agile tools, or even an entire agile tool suite, and then force their staff to use them that the tools will somehow make their teams agile.  The actual result is that you end up with a “standardized” approach to agile that is both overly complex (the tools typically include far more features than you’ll ever want, many of which provide functionality completely inappropriate for your situation) and incomplete (you can’t automate everything and even if you could the vendors haven’t gotten to it yet).  The worst offenders are the Application Lifecycle Management (ALM) suites, many of which seem to be offered by vendors who themselves are struggling to deliver consumable products into the marketplace.
5. Focus on people and process. A common agile coaching refrain is to address tooling once you understand the process, which is good advice to an extent, but it quickly falls down when you realize that many technical practices such as continuous integration (CI) and automated testing require adoption of tooling from the very start. When you ignore or at least greatly downplay tooling issues you tend to grow “agile purists”, and even “agile zealots”, who only know how to apply agile in simple contexts.  Ignoring tools can ease your agile transformation at first but eventually tends to get you into trouble when you start to apply agile in more complex situations.
6. Focus on process and tools. When you don’t coach people in the skills and mindset around
   “being agile” you tend to end up with cargo-cult agile with automated bureaucracy.  Agile transformation requires a paradigm shift, which inherently implies you need to address what it means to be agile.
7. Focus on tools and people. When you downplay how to “do agile” you tend to get agile children playing with shiny new toys.  They tend to know the agile language and mindset, and will often have a cursory understanding of simple agile practices that they learned on their two-day certified mastery workshop, but they really don’t know how to successfully build consumable solutions from beginning to end.  Teams who receive little help on “doing agile” tend to spend a lot of time, and a lot of money, figuring out the agile process.  Worse yet, they often adopt a WaterScrumFall approach where Inception and Transition tends to be more traditional and heavy and only Construction is lightweight and agile.
8. Focus on people, process, and tools simultaneously.  When your agile transformation strategy addresses all three factors at once you have the potential to create Disciplined Agile teams that can work at scale.  This only happens when your “being agile” efforts help people to shift to an agile mindset, when your “doing agile” efforts teach a comprehensive yet flexible (think goal driven) view of agile strategies and practices, and your “supporting agile” efforts lead to a context-driven, enterprise aware approach to tool selection.

In your agile transformation you will spend much more effort addressing people-oriented (being agile) issues than you will either of process (doing agile) or tooling (supporting agile) issues.  Think of it like this: these three factors are effectively the legs of a stool, if you don’t address all three then your agile transformation will fall over.

If you’d like help with your agile transformation, please contact us via ScottAmbler.com.

Many organizations struggle to transition to a more agile or lean way of working.  In this blog posting we address three questions:

1. What factors should your agile transformation address?
2. How important are these factors in practice?
3. How difficult are these factors to address?

What factors should your agile transformation address?

Our experience is that successful agile transformations need to address three fundamental issues:

1. People (being agile).  This factor addresses issues such as individual mindset, team and organizational culture, and team and organizational structure.  As a Disciplined Agile coach you must help people to evolve to an agile mindset, learn new skills, adopt improved collaboration strategies, and evolve the way they are organized to reflect their new ways of working.  As you can see below, this factor typically comprises between 80 to 85% of your agile transformation effort.
2. Process (doing agile).  As a Disciplined Agile coach you need to help organizations adopt new agile and lean practices, strategies, and the Disciplined Agile (DA) toolkit itself.  This tends to be between 10 to 15% of the overall transformation effort.
3. Tools (supporting agile).   Agile teams will need to adopt new tools, such as continuous integration (CI) tools, testing tools, and perhaps even agile task board software to name a few.  Agilists will use existing tools, such as their configuration management environment, in new ways.  And they will abandon some existing tools, in particular traditional test management tools, that aren’t applicable in the agile world.  Reworking your tooling strategy tends to take about 5 to 10% of your agile transformation efforts.

How important are these factors in practice?

In the 2014 Agile Transformation survey we asked a series of questions around how important it was to address various people, process, and tooling factors.  The survey respondents had either been through an agile transformation or were currently well into one.  The figure below shows that the first and third most important transformation factors were aligned with being agile, the second and fourth most important factors were around doing agile, and the two least important factors were around tooling (supporting agile).

How difficult are these factors to address?

In the 2014 Agile Transformation survey we asked a similar series of questions around how difficult organizations found it to address various people, process, and tooling factors.  The first and third most difficult factors to address were cultural (being agile).  The second and sixth most difficult factors to address were process oriented (doing agile) in nature.  The fourth and fifth most difficult factors addressed tooling.

Our experience is that if you don’t address all three factors in your agile transformation effort that you will run into serious trouble.  This topic will be explored in our next blog posting.

If you’d like help with your agile transformation, please contact us via ScottAmbler.com.

This posting, the latest in a series focused on a disciplined agile approach to data management, overviews the activities that a disciplined agile data management team may perform. The Disciplined Agile (DA) toolkit promotes an adaptive, context-sensitive strategy to data management.  DAD does this via its goal-driven approach that indicates the process factors you need to consider, a range of techniques or strategies for you to address each process factor, and the advantages and disadvantages of each technique.  In this blog we present the goal diagram for the Data Management process blade and overview its process factors.

The following diagram overviews the potential activities associated with disciplined agile data management.

The process factors that you need to consider for data management are:

1. Improve data quality.  There is a range of strategies that you can adopt to ensure data quality.  The agile community has developed concrete quality techniques – in particular database testing, continuous database integration, and database refactoring – that prove more effective than traditional strategies.  Meta data management (MDM) proves to be fragile in practice as the overhead of collecting and maintaining the meta data proves to be far greater than the benefit of doing so.  Extract transform and load (ETL) strategies are commonplace for data warehouse (DW) efforts, but they are in effect band-aids that do nothing to fix data quality problems at the source.
2. Evolve data assets.  There are several categories of data that prove to be true assets over the long term: Test data that is used to support your testing efforts; Reference data, also called lookup data, that describes relatively static entities such as states/provinces, product categories, or lines of business; Master data that is critical to your business, such as customer or supplier data; Meta data, which is data about data. Traditional data management tends to be reasonably good at this, although can be heavy handed at times and may not have the configuration management discipline that is common within the agile community.
3. Ensure data security.  This is a very important aspect of security in general.  The fundamental issue is to ensure that people get access to only the information that they should and that information is not available to people who shouldn’t have it.  Data security must be addressed at both the virtual and physical levels.
4. Specify data structures.  At the enterprise level your models should be high level – lean thinking is that the more complex something is, the less detailed your models should be to describe it.  This is why it is better to have a high-level conceptual model than a detailed enterprise data model (EDM) in most cases.  Detailed models, such as physical data models (PDMs), are often needed for specific legacy data sources by delivery teams.
5. Refactor legacy data sources. Database refactoring is a key technique for safely improving the quality of your production databases.  Where delivery teams will perform the short term work of implementing the refactoring, there is organizational work to be done to communicate the refactoring, monitor usage of deprecated schema, and eventually remove deprecated schema and any scaffolding required to implement the refactoring.
6. Govern data.  Data, and the activities surrounding it, should be governed within your organization.  Data governance is part of your overall IT governance efforts.

Looking at the diagram above, traditional data management professionals may believe that some activities are missing.  These activities may include:

• Enterprise data architecture.  This is addressed by the Enterprise Architecture process blade.  The DA philosophy is to optimize the whole.  When data architecture (or security architecture, or network architecture, or…) is split out from EA it often tends to be locally optimized and as a result does not fit well with the rest of the architectural vision.
• Operational database administration.  This is addressed by the Operations process blade, once again to optimize the operational whole over locally optimizing the “data part.”

Future blog postings in this series will explore the workflow associated with data management.

Related Resources

• The Data Management process blade
• DevOps Strategies: Data Management
• Disciplined Agile DW/BI: Artifact Creation
• Disciplined Agile DW/BI Teams
• The Enterprise Architecture process blade
• Why Data Management?

According to the Data Management Body of Knowledge, data management is “the development, execution and supervision of plans, policies, programs and practices that control, protect, deliver and enhance the value of data and information assets.”  In our opinion this is a very good definition, unfortunately the implementation of data management strategies tends to be challenged in practice due to the traditional, documentation-heavy mindset. This mindset tends to result in onerous, bureaucratic strategies that more often than not struggle to support the goals of your organization.

Having said that, data management is still very important to the success of your organization.  The Disciplined Agile (DA) toolkit promotes a pragmatic, streamlined approach to data management that fits into the rest of your IT processes – we need to optimize the entire workflow, not sub-optimize our data management strategy.  We need to support the overall needs of our organization, producing real value for our stakeholders. Disciplined agile data management does this in an evolutionary and collaborative manner, via concrete data management strategies that provide the right data at the right time to the right people.

There are several reasons why a disciplined agile approach data management is important:

1. Data is the lifeblood of your organization.  Without data, or more accurately information, you quickly find that you cannot run your business. Having said that, data is only one part of the overall picture.  Yes, blood is important but so is your skeleton, your muscles, your organs, and many other body parts.  We need to optimize the whole organizational body, not just the “data blood.”
2. Data is a corporate asset and needs to be treated as such.    Unfortunately the traditional approach to data management has resulted in data with sketchy quality, data that is inconsistent, incomplete, and is often not available in a timely manner.  Traditional strategies are too slow moving and heavy-weight to address the needs of modern, lean enterprises.  To treat data like a real asset we must adopt concrete agile data quality techniques such as database regression testing to discover quality problems and database refactoring to fix them.  We also need to support delivery teams with lightweight agile data models and agile/lean data governance.
3. People deserve to have appropriate access to data in a timely manner. People need access to the right data at the right time to make effective decisions.  The implication is that your organization must be able to provide the data that an individual should have access to in a streamlined and timely manner.
4. Data management must be an enabler of DevOps.  As you can see in the following diagram, Data Management is an important part of our overall Disciplined DevOps strategy. A successful DevOps approach requires you to streamline the entire flow between delivery and operations, and part of that effort is to evolve existing production data sources to support new functionality.

In future blog postings we will explore the goal diagram of the Data Management process blade and the associated workflow.

Related Resources

• DevOps Strategies: Data Management
• DAMA Data Management Body of Knowledge (DMBoK)
• Disciplined Agile DW/BI: Artifact Creation
• Disciplined Agile DW/BI Teams
• The Enterprise Architecture process blade