Project Management

Disciplined Agile

by , , , , , , ,
#ChooseYourWoW | #ContinuousImprovement | #Kaizen | #ProcessImprovement | Adoption | agile | Agile certification | agile transformation | Analogy | Architecture | architecture | book | Business Agility | Certification | Choose your WoW | CMMI | Coaching | Collaboration | Compliancy | Configuration management | Construction phase | Context | Continuous Improvement | COVID-19 | Culture | culture | DAD | DAD discussions | DAD roles | Data Management | database | DevOps | Discipline | disciplined agile delivery | Documentation | DW/BI | Enterprise Agile | Enterprise Architecture | Enterprise Awareness | Essence | estimation | Evolving DA | Experiment | Financial | GDD | Geographic Distribution | global development | Goal-Driven | goal-driven | goals | Governance | Guideline | Improvement | inception | Inception phase | Large Teams | layer | Lean | Lifecycle | lifecycle | Metrics | mindset | News | News and events | Non-Functional Requirements | non-functional requirements | Operations | Outsourcing | People | Philosophies | Planning | PMI | PMI and DA | Portfolio Management | Practices | Principle | Process | process improvement | Product Management | Product Owners | Program Management | Project Management | Promise | quality | Release Management | Requirements | requirements | Reuse Engineering | Risk management | RUP | Scaling | scaling | scaling agile | Scrum | serial | Support | Surveys | Teams | Technical Debt | Terminology | Testing | testing | Toolkit | Transformation | velocity | Workflow | show all posts

About this Blog

RSS

View Posts By:

Scott Ambler
Glen Little
Mark Lines
Valentin Mocanu
Daniel Gagnon
Michael Richardson
Joshua Barnes
Kashmir Birk

Recent Posts

Would you like to get involved with the 20th Anniversary of Agile?

The Four Layers of the Disciplined Agile Tool Kit

The Disciplined Agile Foundation Layer

The Team Lead Role: Different Types of Teams Need Different Types of Leaders

Disciplined Agile is a Hybrid

Bridging the Gap Between Agile and Data


At the Agile 2017 conference this week Lynn Winterboer was kind enough to invite me to her workshop which explored how to apply agile strategies in the data space.  She did a great job of facilitating the group of about 40 people through identifying the challenges currently faced by teams. The main issues that the group explored were:

  • Product ownership and data
  • Agile data architecture
  • Data testing and tooling support
  • How to include data people, and activities, in development

Lynn will soon be blogging about the results so I’m not going to dive into that here.  I suspect that her blog post will be very interesting.

What I’d like to do here is share a few thoughts about what I observed:

  1. The discussion was very healthy. This was a group of very smart people coming from different backgrounds.  Everyone was interested in sharing their experiences and working together to solve some tough problems.
  2. Context counts. “Agile and data” is a big topic.  A few people were dealing with the issue of how to address data issues on application development projects, some were focused on agile data warehousing/business intelligence, and some were focused on complex data analytics.  In our conversations it was very clear that strategies which worked for app development wouldn’t work for analytics, and vice versa.  This is why Context Counts is one of Disciplined Agile’s fundamental principles.
  3. The challenges are tough. Everyone was working in organizations that were struggling with these challenges.  For each of issue we could have spent much longer exploring the potential solution(s).
  4. Every challenge we discussed are solved issues.  I’ve always found it frustrating when people, who are very smart and who have been struggling with a problem for awhile, have clearly never thought to simply google “database testing tools” or “agile architecture” to find out what advice is already out there.  When we discussed testing I even asked why people hadn’t done such as search, and then pointed out that there are a lot of tools available and even a few people maintaining lists of such tools (such as 40+ database testing tools).  All of these challenges, and more, have solutions described by techniques of the Agile Data and Agile Modeling methods from about 15 years ago.  These techniques have of course been adopted, and put into context, by the Disciplined Agile (DA) toolkit and in particular Disciplined Agile Delivery (DAD).
  5. The “factions” behaved differently.  Long ago I pointed out that there’s a cultural impedance mismatch between the data and developer communities, and it was pretty easy to observe during this workshop.  For example, during the workshop we were asked to identify solutions to the challenges.  Lynn wanted us to write this information on flip-chart paper so that she could later scan it and share it with others.  For awhile the groups dominated by data people discussed the solutions without writing anything down.  Their conversations were good, but they quickly got stuck in analysis paralysis mode because they seemed to be afraid to write down information for fear that they couldn’t easily update it (the challenge with having paper to write on instead of whiteboards). The groups dominated by agile people, the ones focused on development and architecture, started by writing on sticky notes and putting them on the flip chart paper.  This fundamental agile modelling strategy enabled them to visualize and share their work transparently, enhancing their conversation and enabling them to move forward easily.
  6. Database (tool) vendors need to up their game. Speaking out tools, database vendors and data warehouse tool vendors really need to up their game.  Here’s a few very harsh questions: Does your database tool vendor or ETL tool vendor offer testing tools that enable both black box and white box database testing?  Answer: very likely no, because their customers don’t demand that of them.  Did your data team even think to ask for such tools?  Answer: very likely not.  How many database testing books do you think have been written? Answer: very few, go do a search and see for yourself.  Why does the data community have such a huge blind spot when it comes to testing? Answer: this is a huge side effect of the cultural impedance mismatch.

I’m very happy to see that Lynn is actively trying to bridge the agile and data communities, helping us to learn from each other.  This is something she’s been doing for years and doing it quite well.  My experience is that both communities would benefit greatly from more collaboration with each other.

Posted by Scott Ambler on: August 11, 2017 08:37 AM | Permalink | Comments (0)

Teal is the New Black

Just as your process must be flexible and adaptive, so must your organization. In Reinventing Organizations Frederick Laloux works through the history, and arguably a maturity model, for organizational design. The premise, which is overviewed in the diagram below (you can click on it for a high-res version), is that over time we’re seeing organizations evolve from tribal and often violent structures (Red) through more formalized hierarchical structures (Amber/Orange) to agile approaches (Green/Teal). Today the vast majority of organizations, believed to be 80-90%, are somewhere on the Amber through Green scale.

Laloux Teal Organizations

There are several important observations we’d like to make about Laloux’s organizational maturity scale:

  1. For your organization to support agile it should at least be (mostly) Green, with a participative and values-based culture, or better yet Teal with a truly adaptive and aware strategy (as we’ve been preaching throughout this chapter).
  2. Your organization will start its improvement journey wherever it currently is on the scale. Laloux’s model provides insights into what your current challenges are likely to be and what potential improvements you should consider making.
  3. Teams can still be agile within Orange and Amber organizations, but the organization itself will struggle with agility due to cultural impediments.
  4. It is difficult to jump organizational levels – in other words if your organization is currently Amber then you need to move through Orange, then Green, and finally Teal.
  5. To move between levels you need the both top-down support from leadership as well as bottom-up support from front-line staff – bottom up, “stealth” improvement efforts will fail on their own when organizational antibodies fight back.

You Want to be At Least Green

Why does your organization need to be at least Green or Teal to become agile? Green organizations support a participative culture style that enables collaboration within and between teams. Green organizations explicitly align people around common values, so injecting any missing agile and lean philosophies often proves to be straightforward. Teal organizations go one step further and bring it all together by explicitly recognizing that they are complex adaptive systems (CASs). This provides an environment where agile teams are able to experiment, learn, collaborate, and most importantly thrive as they find new ways to delight their customers.

Improving Horizontally is Much More Realistic

Laloux himself is very clear about the importance of top-down support if you want to become a Teal organization, or frankly just to move up the hierarchy (say from Red to Amber).  In chapter 3 of Reinventing Organizations he states that for an organization to become Teal that it requires the support of both the CEO/founder and the owners of the company.  Our experience is that a third factor is required, the support of the front-line staff (and better yet middle management), for your transformation to be successful.

Laloux believes that it’s much easier for organizations to improve horizontally – become the best Orange or Amber organization that you can be.  In many ways this is much more conducive to a lean continuous improvement strategy than an “agile transformation” strategy.

Your Organization is Probably a Rainbow

It’s attractive to think that your organizational culture is consistent across the entire enterprise, but it is far more likely that you have teams or divisions with differing color ratings according to Laloux’s model. This is because the culture of a team (or division) is greatly influenced by the leader of that team, and leaders vary in their style, and because teams face unique situations – sometimes a red strategy is the most appropriate given what the team faces. Context counts!

Suggested Reading

Posted by Scott Ambler on: July 01, 2017 05:50 AM | Permalink | Comments (0)

7 "Easy" Steps For Convincing Senior Management to Support a Hard Change

Puzzled manager

We’re often asked how do you convince senior management to accept new agile ideas and strategies.  Examples of such ideas include:

Why is This Important?

There are three reasons why it’s important for agile coaches, and Team Leads/ScrumMasters for that matter, to know how to convince senior management to support new ways of working:

  1. You need their support.  There are many aspects of an agile transformation that an agile coach doesn’t have the authority nor the resources to change.  As a result you need to collaborate with others and very often earn the support of the people who do have the authority and resources.
  2. Agile transformations are complex.  Agile transformations are about more than just transforming software development teams.  The 2016 Agility at Scale study found that 96% of agile teams need to collaborate with other teams external to them to get their job done.  The implication is that these other teams that they are collaborating with – including your business stakeholders, governance team, data management, internal audit, security group, and many others – need to at least be able to interact with your agile teams in a reasonably flexible manner if not work agilely themselves.
  3. Agile transformations are fragile.  If you want to transform your IT department, and more importantly your organization, then you’ll need to transform all aspects of the department/organization.  All it takes is one or more groups to refuse to work in an agile manner and suddenly your transformation is at risk.  The implication is that you need to get good at convincing others to support your efforts if not change themselves.

Why is This So Hard?

There are many reasons why senior management may be reticent to consider this change that you believe to be very important:

  • They have other priorities that you may not be aware of.
  • They have many other issues to deal with, this is just one of them.
  • They may be very happy with the status quo and don’t recognize the problem.
  • There are other people advising the exact opposite.
  • There are people who are entrenched in the existing way of working, and that may include senior management.
  • They will need to convince their peers regarding the benefits of the change and they may not know enough to be able to do so, or may not have the political capital to effect the change.
  • Change can be disruptive and it may jeopardize their existing commitments which incidentally might be tied to their compensation plan.
  • The manager realizes that this change has greater ramifications than you may believe.

The Seven “Easy” Steps For Convincing Someone to Support Change

Here is an approach that we’ve had work in practice for us.  You will very likely need to work through all of these steps, pretty much in order, to be successful.  These steps are:

  1. Pick your battles wisely.  Ask yourself whether this issue is the most important thing that you need help with from this person.  There will be only so much willingness to invest time and effort in supporting the changes that you believe to be made, and not all requests are going to be supported.  As Rod Bray, CDAC, likes to say: “Choose the hill that you’re willing to die on.”
  2. Know the topic and the language around it.  Chances are that you will need to be able to explain whatever it is that you’re asking for help on, what the trade-offs are, why its better than the current approach, and what the impact of the change will be.  To do this you will need to understand the trade-offs are of the current approach and understand the issues and language of the topic.  For example, if you’re asking for help to change the way that IT projects are funded, are you able to speak intelligently about the existing annual-based budgeting process, project-based funding, and perhaps even the implications of CAPEX/OPEX?  Or, if you’re asking to improve the current approach to IT governance, do you understand the existing governance process, what it’s trade-offs are, and what the potential impacts of applying traditional governance to agile teams may be?  If you don’t have this fundamental understanding of the topic then you will very quickly sound like you don’t know what you’re talking about, so why would management want to support you?
  3. Plan the conversation.  Although you very likely have some very great ideas, if you spring them on others they will very likely be threatened by them at first (human beings are psychologically wired to treat surprises as threats).  A better approach is to first ask permission to discuss a new idea that you have, and even share an overview of the idea beforehand so that they can think about it a bit, before you get together to seek their support.
  4. Explore their concerns.  Once you’ve pitched your idea to them they will very likely want to discuss the trade-offs with you, in particular the impact on other groups and the time and effort required to support your change.  The implication is that part of your preparation before you make your pitch should be to think about what concerns they may have with your suggested approach so that you have arguments to counteract any concerns.
  5. Ask them to share their actual experiences.  It is very common for people to become attached to ineffective ways of working.  This sounds strange on the surface, but people are like this.  Whenever we run into someone who believes in a strategy that we know to be ineffective – fixed price funding, documentation-based governance, detailed up-front modeling, significant amounts of manual testing to name a few – we ask them how well it’s working for them in practice.  Very often they’ll tell you about the positives, but if you know the topic (and better yet the history of that strategy within your organization) then you can start exploring the negative aspects with them too.  It’s particularly useful to be able to bring up several past projects where that strategy was applied yet it didn’t work out so well in practice. The point is to help them to recognize that their favored strategy isn’t working as well as they’d like, and that there is a need to rethink your current approach.
  6. Educate them.  Walk them through the trade-offs, both good and bad, of your suggested approach.  Be prepared to discuss the trade-offs of the current strategy, and in particular relate those trade-offs back to the experiences that they just told you about.  You may often discover that they didn’t realize that there are other options available to them and that they’ve been ignoring the problems with their existing approach.  Help them to understand that they have a better choice available to them.
  7. Convince them to run a small experiment.  Making a large, whole-scale change is scary.  If the new approach doesn’t work out then you’ve got a serious problem to deal with and the manager who sponsored the change may be punished for it.  But, running a small, contained experiment to see if the new strategy works in your environment isn’t very threatening and better yet is a fundamental risk management strategy.  So start small, get a visible win, learn from the experiment, and the roll out the change more widely.  It is important that you “negotiate” the changes as they will be more likely to try it if you let them know that the change is an experiment and they will have the opportunity to revert back if the expected benefits do not materialize.  Note that some organizations are leery of running “experiments” but are very willing to run “proof of concepts (PoCs)” – go with the terminology that works in your organization.

We wish that we could tell you that we’ve had a 100% success rate with this strategy.  Sadly we haven’t.  We have done very well with this, but sometimes it doesn’t always work out the first time.  Or the second time, and sadly sometimes not even the third time.  Your goal should be to get them thinking about new ways of working and to give them the time that they need to decide to support you.

Posted by Scott Ambler on: March 22, 2017 06:18 PM | Permalink | Comments (0)

Adopting a Disciplined DevOps Strategy

Categories: Adoption, Culture, DevOps

In this continuing series about how the Disciplined Agile (DA) toolkit supports and enhances DevOps, we overview several strategies to help your organization adopt a Disciplined DevOps strategy:

  1. Invest in your people.  In our experience 80 to 90% of your overall effort will be in helping people to learn new skills and ways of thinking and to rethink if not abandon many of the “best practices” of yesteryear.  This requires training, education, and coaching over a long period of time – most people will require many months, and sometimes years, to make the transition to this new mindset.
  2. Create a safe learning environment.  Teams must be free to experiment, to try new strategies to discover what works for them in the situation that they face.  Very often this will work out well, with a few stumbles along the way, but every so often the experiment will show that the strategy just isn’t right for this team.  That should still be considered a successful experiment, learning what doesn’t work is just as valuable as learning what does, and the team should not be worried about recrimination for “failed” experiments.
  3. Look at the “whole system”.  Disciplined DevOps is about more than just continuous delivery (although that is a great start) and in most enterprises it’s about more than just streamlining development and operations.  With a Disciplined DevOps mindset we strive to improve flow through the entire ecosystem, including development, operations, support, enterprise architecture, data management, release management, and most importantly to the business itself.
  4. Improve locally, transform globally.  Each team, including your solution delivery teams, your enterprise architecture team, your operations staff, and many others must strive to improve and streamline the way that they work.  These local improvement efforts must be supported by a “global” transformation effort that focuses on improving DevOps across your entire ecosystem.  Every team will affect other teams, motivating them to make improvements which in turn affects how they work with others.  Your IT department is a complex adaptive system where people and teams learn and improve over time in a dynamic, evolutionary manner.  If you just focus on local improvements your DevOps effort is likely to devolve into chaos.  If you just focus on a company-wide, global transformation it is likely to get bogged down in bureaucracy.  An “improve locally, transform globally” approach is a viable middle ground that benefits from these two extremes while avoiding the disadvantages.
  5. Have a communication plan (and work it).  Adopting a Disciplined DevOps strategy within your organization typically requires a lot of (often small) changes.  Although it may be clear to you why this is important it isn’t always clear to everyone else.  People need to understand why you’re making these changes, what’s in it for them, what the overall change strategy is, how far along the plan you currently are, what changes are coming soon, and so on.  You communication plan may include regular newsletters, posters overviewing key concepts, brown bag lunches where people share their experiences, electronic discussion forums, management presentations, and many more.  The keys to success are to have a constant drum beat of information, to be as open and honest about what is actually occurring, to provide opportunities to everyone to learn, and to motivate everyone to share their learnings.
  6. Think long-term.  Disciplined DevOps is a journey, not a destination.  It takes time for people to adopt a new mindset, months and often years before it is truly ingrained in their way of thinking.  This paradigm shift does not occur by management fiat, nor does it occur as the result of a day or two of training (although training is important), nor does it occur because you’ve invested in new tooling.  Organizations that successfully make this paradigm shift do so by investing in their people, their process, and their tooling over the long term.
Posted by Scott Ambler on: April 26, 2015 07:32 AM | Permalink | Comments (0)

Does your team own its process or merely rent it?

Process ownership

An important philosophy within both the agile and lean communities is that a team should own its process. In fact, one of the principles behind the Agile Manifesto is “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.” The idea is that teams should be empowered to tailor their approach, including both their team structure and the process that they follow, to meet the unique needs of the situation that they find themselves in. Teams that own their process will tailor it over time as they learn how to work together, adopting new techniques and tweaking existing ones to increase their effectiveness.

As with most philosophies this one is easy to proselytize but not so easy to actually adopt. When it comes to process improvement, teams will exhibit a range of behavior in practice. Some teams see process as a problem and actively seek to ignore process-related issues. Some teams are ambivalent towards process improvement and generally stick with what they’ve been told to do. And some teams see process improvement as an opportunity to become more effective both as a team and as individuals. This range of behaviors isn’t surprising from a psychology point of view although it can be a bit disappointing from an agile or lean point of view. It has led me to think that perhaps some teams choose to “own” their process but many more still seem to prefer to simple rent it.

The behaviors of people who rent something are generally different than those who own something. Take flats for example. When you rent a flat (called an apartment in North America) you might do a bit of cosmetic work, such as painting and hanging curtains, to make it suitable for your needs. But people rarely put much more effort than that into tailoring their rental flat because they don’t want to invest money in something that isn’t theirs, even though they may live in the flat for several years. It isn’t perfect but it’s good enough. When you own a flat (called a condo in North America) you are much more likely to tailor it to meet your needs. Painting and window dressings are a good start, but you may also choose to renovate the kitchen and bathroom, update the flooring, and even reconfigure the layout by knocking down or moving some walls. One of the reasons why you choose to own a flat is so that you can modify it to meet your specific needs and taste.

You can observe similar behaviors when it comes to software process. Teams that are merely “process renters” will invest a bit of time to adopt a process, perhaps taking a two-day course where they’re taught a few basic concepts. They may make a few initial tailorings of the process, adopt some new role names, and even rework their workspace to better fit the situation that they face. From then on they do little to change the way that they work together. They rarely hold process improvement sessions such as retrospectives, and if they do they typically adopt changes that require minimal effort. Harder improvements, particularly those requiring new skills that require time and effort to learn, are put off to some point in the distant future which never seems to come. Such behavior may be a sign that this “team” is not even be a team at all, but instead a group of individuals who are marginally working together on the same solution. They adopt the trappings of the method, perhaps they spout new terminology and hold the right meetings, but few meaningful changes are actually made.

Process owners behave much differently. Teams that own their process will regularly reflect on how well they’re working and actively seek to get better. They experiment with new techniques and some teams will even measure how successful they are implementing the change. Teams that are process owners will often get coaching to help them improve, both at the individual and at the team level. Process owners strive to understand their process options, even the ones that are not perfectly agile or lean, and choose the ones that are best for the situation they find themselves in.

The Disciplined Agile (DA) toolkit is geared for teams that want to own their process. The DA toolkit is process goal-driven, not prescriptive, making your process choices explicit and more importantly providing guidance for selecting the options that make the most sense for your team. This guidance helps your team to get going in the right direction and provides options when you realize that you need to improve. DAD also supports multiple lifecycles because we realize that teams find themselves in a range of situations – sometimes a Scrum-based lifecycle makes sense, sometimes a lean lifecycle is a better fit, sometimes a continuous delivery approach is best, and sometimes you find yourself in a situation where an exploratory (or “Lean Startup”) lifecycle is the way to go.

You have choices, and DAD helps guide you to making the choices that are right for you in your given context. By providing process guidance DAD enables your team to more easily own its own process and thereby increase the benefit of following agile or lean approaches.

Posted by Scott Ambler on: March 03, 2014 07:37 AM | Permalink | Comments (1)
ADVERTISEMENTS

"I've had a perfectly wonderful evening. But this wasn't it."

- Groucho Marx