Disciplined Agile

In my experience in running dozens of process tailoring workshops, over several years, with of teams of every shape size and experience level and in different organizations, the most recurring comment is that the workshops “revealed all kinds of options we didn’t even realize were options!”   Although almost always a bit of a hard sell at the outset, I have yet to work with a team unable to quickly grasp and appreciate the value of these activities.

I had to do quite a bit of experimenting in order to get the timing and content of the workshops right – and learned over time that success is also predicated on knowing whom to include when. My first attempts were gruelling, close-to-full day affairs with entire teams in attendance, held at or close to project kickoff. Though transparent and inclusive, to my surprise this approach was actually deemed a waste of their time by many team members, especially those whose contribution would occur primarily in the construction phase. First lesson learned – A technical team lead, architect or senior developer can actually stand in for most of the developers in the early stages. I find it helpful to always bear in mind what George Dinwiddie (http://www.velocitypartners.net/blog/2014/02/11/the-3-amigos-in-agile-teams/) dubbed “the 3 amigos” in determining who needs to attend a process tailoring session. Be it at inception, construction, or even in transition, you need to tailor not only the processes, but also the attendance of the workshop in order to ensure you have the right mix of people, with the right collaborative mindset, to cover issues pertaining to 1) the business problem being addressed  2) the potential technical solutions to that business problem and 3) the processes (both team and organizational) that will enable the work to be carried out.

My second lesson learned pertained to the format and presentation of the process blades themselves. I found that simply working from the published process maps was insufficient, as we ran into onerous issues around how to best record the WoW choices teams were making. I eventually reproduced the entire process blade library in a spreadsheet format, with columns for comments. This seemingly innocuous administrative step quickly ushered in the third lesson learned – the sessions can be used not merely to document an immediate WoW decision, but also to identify future, more “mature” aspirational choices which the team can set as goals over a specified time period.

A fourth lesson learned, and one that was also enabled by using a simple spreadsheet tool, is that it became far easier to Align with Enterprise Direction. By “locking down” enterprise-level process choices across all the blades where applicable, a lot of potentially fruitless (at that point in time) discussion was saved for many a team. No use in discussing test automation strategies to death for instance in divisions still completely relying on manual tests, or at the opposite end of the spectrum, teams endowed with high-performing, well-integrated CI/CD environments. This is a large part of what DA calls self-organization within appropriate governance.

The fifth and final major lesson learned was to never start from a blank slate if at all possible. I would typically show up at a team’s first process tailoring workshop with a pre-filled version from another team facing somewhat similar challenges (the identifying data being scrubbed so they could not identify the previous team). I would then challenge the new team to reflect on the choices and determine whether they made sense for their own context. This also saved time and effort, as there are recurring themes and common challenges within organizations that all teams face.

Here’s an important note on determining participation – Ultimately, the teams themselves are the best arbiters of who should attend the sessions at varying stages of advancement. Allowing this will typically result in a bit of initial over participation, followed by under participation (especially is the pressure is on to get “real” work done!) – the key as facilitator is to coax the team back into balanced participation, and to lobby the organization for the necessary support in freeing people up. The support will become easier and easier to obtain as the benefits of allowing teams to choose their WoW become apparent.

Finally, be prepared for surprises. I once ran through the Program process blade with a team, only to have them come to the realization that … they weren’t really a Program! Which was actually a good thing as it helped avoid introducing a considerable amount of overhead, particularly in the area of program-level KPIs.

This posting, the latest in our series focused on a disciplined agile approach to continuous improvement, overviews the activities associated with it. The Disciplined Agile (DA) toolkit promotes an adaptive, context-sensitive strategy.  The framework does this via its goal-driven approach that indicates the process factors you need to consider, a range of techniques or strategies for you to address each process factor, and the advantages and disadvantages of each technique.  In this blog posting we present the goal diagram for the Continuous Improvement process blade and overview its process factors.

The following process goal diagram overviews the potential activities associated with disciplined agile continuous improvement. These activities are performed by, or at least supported by, a process improvement team (sometimes referred to as a Software Engineering Process Group, or SEPG).

The process factors that you need to consider for continuous improvement are:

1. Identify improvements.  There are several ways that your process improvement group can support the identify of potential improvements within your organization.  One of the more effective strategies is to help teams adopt the practice of holding regular retrospectives.  Although it is common for disciplined agile delivery teams to hold retrospectives this is often a new concept for enterprise architecture teams, IT governance teams, data management teams, and so on.  We also get very good traction with value stream mapping and brainstorming sessions, which invariably proves to be far more effective than the traditional approach of creating current and proposed (business) process models that rarely seem to result in lasting acceptance of the proposed new way of working.
2. Share improvements.  As you can see there are multiple ways that you can share improvement ideas between teams, many of them being free or at least very inexpensive to implement.  We’ve had very good experiences with internal discussion forums such as Jive or ActiveBoard, practitioner presentations (often called lunch and learns) where someone presents  their learnings to others, lean coffee sessions where people voluntarily meet at a regular time to share ideas, and communities of practice (sometimes called guilds) who purposely collaborate to educate themselves in a given topic.
3. Capture improvements.  There are various ways that identified improvements may be captured to retain them over time.  Possible strategies include describing each learning in a document and then managing that document in a documentation repository such as Sharepoint or more simply in a shared folder; capturing the learnings in a shared wiki such as Confluence; describing your evolving process using a process repository such as Stages or Rational Method Composer; or via an expert system such as Enterprise Transformation Advisor.
4. Support teams. Your process improvement team can help others to adopt process improvement techniques through training, education, and coaching.  They can also facilitate team assessments and retrospectives (a great idea is to co-facilitate a few retrospectives with someone on the team to transfer those skills to them).  A very effective strategy is to help a team run a process improvement experiment or two, particularly in situations where the team isn’t being supported by a coach.  This helps them see that they can safely try new ideas within their environment for a few iterations to determine whether it works for them or not.  Many teams, particularly those new to agile, often do not feel empowered to run such experiments and thus need help to do so.
5. Organize Communities of Practice (CoPs).  A Community of Practice (CoP) is a collection of people who share a craft or profession who have banded together to ‘learn’ from each other to develop themselves and often even the  organization.  We’ve seen CoPs for testing, architecture, agile/lean, business analysis, technical debt, and many others.  CoPs will often perform the activities called out by the Identify Improvements, Share Improvements, Capture Improvements, and Support Teams process factors.  A CoP will often start up when one or more practitioners within your organization recognize the need for it, although sometimes it may also start to support the efforts of a corresponding Center of Excellence (CoE).  Participation in a CoP is typically voluntary.
6. Organize Centers of Excellence (CoEs).  A Center of Excellence (CoE) is a group of people with specialized skills and expertise whose job is to provide leadership and purposely disseminate that knowledge within your organization.  CoEs are often created by an organization to support the adoption of a new technology or technique, and in fact  the creation of an Agile CoE is often a key component of your organization’s overall Agile transformation efforts.  Over the years we’ve seen CoEs for object technology (particularly in the 90s when it was new to most companies), solution architecture, testing automation, and of course agile/lean.  Participation as a member of a CoE will be part of, or the entire job for someone.
7. Govern improvement.  It is very common for senior management to want to know whether or not the organization is benefiting from your investment in adopting agile and lean techniques (or other potential improvements for that matter), how much things are improving, and how widespread the adoption is.  The implication is that there needs to be some way to monitor and report on, preferably in a lightweight and streamlined manner, the improvement activities.

Future blog postings in this series will explore the workflow between continuous improvement and other process blades as well as the internal workflow of a process improvement team.

Retrospectives are a great way for teams to explore potential improvements to the way that they work.  A team will get together, discuss what is working well for them, what is not working so well, and hopefully identify ways that they could improve.  It’s this last activity that can be challenging.  You may know that your team is facing a problem but you might not understand your options.  For example, perhaps your team is struggling with the way that it is being funded.  The current funding mechanism is to estimate the cost up front and then allocate these funds to your team.  This motivates your team to be wary of changing requirements due to the fear of going over budget, something that decreases your ability to produce a solution that meets the true needs of your stakeholders.  You have suggested to management several times that a time and materials (T&M) approach would be more appropriate, but you have gotten nowhere with that conversation.

This is where DAD’s process goal-driven approach can help out.  In this case the goal Secure Funding provides some insight.  The process goal diagram, see below, along with the supporting descriptions of each technique, their advantages and disadvantages, and advice for when the technique is applicable can help your team to understand their options and hopefully argue for a better funding strategy.  Although a T&M approach might not be palatable to your financial team right now, perhaps they would be willing to consider a stage gate approach to funding.  Or, perhaps they would be open to a T&M approach but they just don’t understand the tradeoffs between T&M and fixed cost.  With DAD’s goal-driven approach the team can arm itself with the arguments that it needs to have a knowledgeable conversation with the actual decision makers.

Of course this is just one example.  The DA toolkit addresses a range of goals pertinent to successful agile solution delivery, all of which can provide team’s insight into potential process improvement options.  Knowing your options is an easy way to up your game during retrospectives.

An important philosophy within both the agile and lean communities is that a team should own its process. In fact, one of the principles behind the Agile Manifesto is “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.” The idea is that teams should be empowered to choose their way of working (WoW), to "own their process," including both their team structure and the process that they follow, to meet the unique needs of the situation that they find themselves in. Teams that own their process will tailor it over time as they learn how to work together, adopting new techniques and tweaking existing ones to increase their effectiveness.

As with most philosophies this one is easy to proselytize but not so easy to actually adopt. When it comes to process improvement, teams will exhibit a range of behavior in practice. Some teams see process as a problem and actively seek to ignore process-related issues. Some teams are ambivalent towards process improvement and generally stick with what they’ve been told to do. And some teams see process improvement as an opportunity to become more effective both as a team and as individuals. This range of behaviors isn’t surprising from a psychology point of view although it can be a bit disappointing from an agile or lean point of view. It has led me to think that perhaps some teams choose to “own” their process but many more still seem to prefer to simple rent it.

The behaviors of people who rent something are generally different than those who own something. Take flats for example. When you rent a flat (called an apartment in North America) you might do a bit of cosmetic work, such as painting and hanging curtains, to make it suitable for your needs. But people rarely put much more effort than that into tailoring their rental flat because they don’t want to invest money in something that isn’t theirs, even though they may live in the flat for several years. It isn’t perfect but it’s good enough. When you own a flat (called a condo in North America) you are much more likely to tailor it to meet your needs. Painting and window dressings are a good start, but you may also choose to renovate the kitchen and bathroom, update the flooring, and even reconfigure the layout by knocking down or moving some walls. One of the reasons why you choose to own a flat is so that you can modify it to meet your specific needs and taste.

You can observe similar behaviors when it comes to software process. Teams that are merely “process renters” will invest a bit of time to adopt a process, perhaps taking a two-day course where they’re taught a few basic concepts. They may make a few initial tailorings of the process, adopt some new role names, and even rework their workspace to better fit the situation that they face. From then on they do little to change the way that they work together. They rarely hold process improvement sessions such as retrospectives, and if they do they typically adopt changes that require minimal effort. Harder improvements, particularly those requiring new skills that require time and effort to learn, are put off to some point in the distant future which never seems to come. Such behavior may be a sign that this “team” is not even be a team at all, but instead a group of individuals who are marginally working together on the same solution. They adopt the trappings of the method, perhaps they spout new terminology and hold the right meetings, but few meaningful changes are actually made.

Process owners behave much differently. Teams that own their process will regularly reflect on how well they’re working and actively seek to get better. They experiment with new techniques and some teams will even measure how successful they are implementing the change. Teams that are process owners will often get coaching to help them improve, both at the individual and at the team level. Process owners strive to understand their process options, even the ones that are not perfectly agile or lean, and choose the ones that are best for the situation they find themselves in.

The Disciplined Agile (DA) tool kit is geared for teams that want to own their process. The DA tool kit is process goal-driven, not prescriptive, making your process choices explicit and more importantly providing guidance for selecting the options that make the most sense for your team. This guidance helps your team to get going in the right direction and provides options when you realize that you need to improve. DA also supports multiple life cycles because we realize that teams find themselves in a range of situations – sometimes a Scrum-based life cycle makes sense, sometimes a lean life cycle is a better fit, sometimes a continuous delivery approach is best, and sometimes you find yourself in a situation where an exploratory (or “Lean Startup”) life cycle is the way to go.

You have choices, and DA helps guide you to making the choices that are right for you in your given context. By providing process guidance DAD enables your team to more easily own its own process and thereby increase the benefit of following agile or lean approaches.