Disciplined Agile

In Time Tracking and Agile Software Development we overviewed why teams should consider tracking their time.  Primary reasons include:

• You’re billing your customer by the hour
• Your organization wants to account for CapEx/OpEx
• Your organization wants to take advantage of tax credits (typically for R&D work)

A secondary reason to track time is because the team wants to measure where they are spending time so as to target potential areas to improve.  This is more of a side benefit than anything else – if this was your only reason to track time you’d be better off simply discussing these sorts of issues in your retrospectives.  But if you were already tracking time then running a quick report to provide the team with intel likely makes sense for you.

So what are your options for recording time?  Potential strategies, which are compared in the following table, include:

1. Automated report from an agile management tool. The basic idea is to extract data from an agile management tool (JIRA, TFS, LeanKit, …) and load it into your time tracking system.
2. Manual input by team members. Each team member, typically once a week, inputs their time into the time tracking tool.
3. Manual input by the Team Lead. The Team Lead (ScrumMaster) inputs the time for their team into the time tracking tool.
4. Manual input by a Project Manager/Coordinator. A PM or Project Coordinator, often in a support role to the team, inputs the time of team.
5. Don’t track time at all. ‘Nuff said.

Table: Comparing options for tracking time.

This blog posting was motivated by a conversation that I had with Stacey Vetzal on Twitter.

Related Reading

• Time Tracking and Agile Software Development
• Agile Software Development and CapEx/OpEx

Key tenets of agile and lean are to work collaboratively and to streamline your workflow respectively. This includes all aspects of your workflow, not just the fun software development stuff that we all like to focus on.  This blog posting explores how Data Management activities fit into your overall process.

In the process flow diagram below we see that Data Management is a collaborative effort that has interdependencies with other DA process blades and the solution delivery teams that Data Management is meant to support. Due to the shortened feedback cycles and collaborative nature of the work this can be very different than the current traditional strategies. For example, with a DA approach, the Data Management team works collaboratively with the delivery teams, Operations, and Release Management to evolve data sources. The delivery teams do the majority of the work to develop and evolve the data sources, with support and guidance coming from Data Management. The delivery teams follows guidance from Release Management to add the database changes into their automated deployment scripts, getting help from Operations if needed to resolve any operational challenges. Evolution of data sources is a key aspect of Disciplined DevOps.

This highly collaborative strategy is very different than the typical traditional strategy that requires delivery teams to first document potential database updates, have the updates reviewed by Data Management, then do the work to implement the updates, then have this work reviewed and accepted, then work through your organizations Release Management process to deploy into production.

In the next blog posting in this series we will explore the internal workflow of a Disciplined Agile approach to Data Management.  Stay tuned!

For those of you who are Star Trek fans you’ve likely been seeing ads for this t-shirt on your social media feeds.  It is an apt metaphor for the empirical approach that we take with Disciplined Agile – we regularly run studies to explore what is actually going on out there on agile teams, we gather data, as opposed to pouting some of the wishful thinking (spreading lore) that we often hear from consultants and vendors.

We are currently running an agile mini-survey of only 6 questions, so this will take you a few minutes at most to fill out, exploring some important issues about agile adoption within your organizations.  We hope that you choose to invest a few minutes of your valuable time to fill it out, and thank you in advance for doing so.

What Will We Do With the Results?

As you already know the surveys that we run are completely open – We share the source data (without identifying information), the questions as they were originally asked, and a Powerpoint deck summarizing our interesting findings after the survey has closed.  In fact, we have the results from dozens of previous studies posted at the IT Surveys page for you to take a look at.

We also write blogs discussing the results.  For example, for the 2016 Agile Scaling survey that we ran in November, we published several blogs:

• Can you outsource and still be agile?
• Do agile teams take on hard problems?
• Do agile teams face regulatory compliance?
• How “whole” are agile teams in practice?
• How geographically distributed are agile teams in practice?
• How large are agile teams in practice?
• Do agile teams pay down technical debt in practice?

Recently, we’ve created a new infographic summarizing the results of the study.  If you click on the thumbnail below it will take you to the page where you can download a high-resolution PDF of it.  This infographic is only available to members of the Disciplined Agile Consortium (DAC).

Where Can You Get the T-Shirt?

If you’re interested in the T-Shirt, it is a time limited offering on Teespring.

I was hoping to come up with a pithy, short answer to this question but the only thing that I can come up with is “people.”  The not-so-pithy answer is that there is no sort of agreement around what it means to be a “qualified agile coach”, the people hiring coaches aren’t thinking things through in many cases, and the agile community suffers from a myriad of integrity challenges when it comes to professionalism. In this blog I work through the following ideas:

1. Why is there a dearth of qualified agile coaches?
2. Sports coaches as an example
3. What should we expect from agile coaches?
4. Our solution: Agile Coaching-Focused Certifications
5. Parting thoughts

Why is There A Dearth of Qualified Agile Coaches?

Let’s answer this in two parts: Why is there a dearth of agile coaches and why are there so many unqualified coaches available?  The first question is very easy to answer.  The demand for agile coaches far outstrips the supply.  The adoption rate for agile has been growing steadily since 2001, hence the growing demand.  As you’ll see later in this blog, it takes years to grow good coaches.  As a result there is little hope for the supply to catch up with demand any time soon.

The second question, why are there so many unqualified coaches available, is easy but uncomfortable to answer.  In general we have systemic challenges in the IT industry and in many ways we’ve managed to exacerbate these problems within the agile community. Some of the challenges within the IT community include:

• A person is just as likely to be a self taught programmer, and more likely in fact, than they are to have a computer science or engineering degree
• There is no sort of apprenticeship culture in this industry
• Few people have a personal goal of mastery, and few organizations support the gaining of mastery amongst their staff
• There is a shortage of talented people, so It is very easy to find and retain employment regardless of your level of talent, and the market is still growing
• No country has a licensing body for software professionals that is commonly required by organizations, unlike other professions such as doctors, lawyers, engineers, architects, and many more
• Many people in the IT community believe this normal for a professional, or if they do perceive a problem they are (rightfully) overwhelmed with the challenge of addressing it
• Colleges and universities are endemically years behind the quickly changing IT industry (there are a handful of schools that work closely with industry, so it’s getting better)

Then we have the agile community, with its various certification training scams.  You can become a certified master after staying awake during a two-day workshop and passing an online test that almost nobody fails.  To put this into context, a Starbucks barista, the kid who pours your morning coffee, get’s three days of training before being let loose on customers.  Yet it somehow makes sense that someone with 50% less training becomes the lead of a software development team?  Really?  Another example: Someone can become a scaling program consultant after attending a four-day workshop, and worse yet are now “qualified” to teach a two-day workshop to others so as to impart their vast agile scaling knowledge upon them.  Amazingly, because of the demand by companies desperate to hire agile-skilled people, the demand for these “designations” is incredible (shameful would be a more appropriate word).

In practice many agile designations are little more than “participation ribbons”, yet most organizations take them seriously often either because they don’t realize how trivial they are to earn or because they’ve given up expecting any better from agilists.  Is it any surprise that it’s hard to find qualified coaches when we’ve watered things down so much?

Sports Coaches as an Example

Coaching is very common in sports and with the exception of “pick up” games few sports teams are without a coach.  In fact, serious sports teams tend to have several coaches, typically lead by a head coach. In professional sports coaches are paid significant salaries, sometimes millions of dollars a year, as coaching is perceived to be a critical success factor.  It makes sense to look at sports coaching works to see how agile coaching might work.

Most sports coaches are former players.  They’ve typically played for years, and sometimes decades, having been coached themselves all along the way.  They’ll often start off as children, in Canada it’s common for kids to start learning to skate and play hockey at the age of two, being coached and drilled in basic skills and knowledge for years.  They also gain practical experience playing games.  Most kids drop out eventually, although many still play their sport (be it hockey, football, cricket, baseball, …) well into middle age.  And some decide to stay in the sport, but make the shift from being a player into being a coach.

The transition to becoming a sports coach generally isn’t easy.  There are three common strategies for this:

1. Formal training.  One path is to go to university, get a teaching degree, and become a gym teacher at a middle school or high school and begin coaching children.  These coaches tend to coach a wide range of sports, although in some cases you’ll often see a coach specialize on a single sport, such as high school football or hockey, because that’s what their passion was a child (and often because they hope to move up the ranks at eventually, see point #3 below).
2. Informal apprenticing.  Another path is to apprentice, asking your existing coach to allow you opportunities to coach others under their guidance.  When I trained in karate this was very common, with senior students helping to teach less senior students.  My daughter is currently learning to skate and they follow a similar pattern with senior skating coaches (adults) being helped by assistants (typically teenagers who have been skating for many years) to coach and teach the younger children.  Helping to teach or coach others is recognized as an important part of your own learning process.
3. Formal apprenticing.  Because of the money involved professional sports teams tend to take a more formal approach to coaching.  They will often expect coaches either come up through the coaching ranks – start as a high school coach, then become a college-level coach, then finally a professional coach – or to come up through the professional sports ranks – when your star players are past their peak they sometimes move into coaching roles.  Each time you move up to a new level of coaching, say from high school football to college football, you often start as an assistant coach to first prove yourself.  The head coach at each level recognizes that it’s their responsibility to grow more coaches, so they impart their skills and knowledge on to the junior coaches.

So, what are some important observations we can make out of all of this?  First, sports coaches have deep skills and experience at the sports that they are coaching.  Second, we expect this of them.  Would you pay to have your child to be given skating lessons by a “Certified Skating Master” who had two days of training in the “skating mindset” and how to facilitate a handful of skating meetings?  Of course you wouldn’t.  Instead you’d want someone who had been skating for years, and better yet may have even been a competitor at some point in the past.  Third, it takes years of apprenticing or training to become a good sports coach, not just several days in a certification workshop.

What Should We Expect From Agile Coaches?

Here is what we’ve found to be the critical success factors for agile coaches:

1. They should have years of agile experience, not days of training.  If someone doesn’t have years of experience in something, and more importantly years of varied experience, then why they heck would you hire them as a coach?
2. They should have coaching skills and experience.  Being experienced in agile isn’t enough.  Apprenticing under another good agile coach is a great way to get coaching skill as is getting training in agile coaching (the program at International Coaching Federation (ICF) is very thorough).  The need for experience is a bit of a catch-22 of course – you need to already be an agile coach to be qualified to be an agile coach.  But, if someone has no previous coaching experience then at best I’d put them into a junior coaching role under the guidance of an experienced coach.  This provides them with the opportunity to gain the requisite experience and to prove themselves in practice.
3. They should have robust agile skills and knowledge.  Years of agile experience is a good start, but better yet is a range of experience at all aspects of the lifecycle in which they are coaching.  It’s reasonable to expect a delivery team coach to understand all aspects of agile solution delivery so that they can coach the entire team in the skills they need to succeed.  Furthermore, it’s reasonable to expect an Agile IT coach to have experience in the full agile IT lifecycle, including areas such as Enterprise Architecture, Data Management, Portfolio Management, and many others.
4. They should have experience in a similar context.  Ideally they should have skills in a similar context to what you currently face – someone who only has small team coaching experience will struggle to coach a large programme, someone who only has only coached in startup companies will struggle in a large financial institution, someone who has only coached co-located teams will struggle with globally distributed teams.  Context counts.

All four of these factors are equally important.  Any “coach” who is deficient in one or more of these areas still has some work to do.  Nobody is perfect of course, given the rates that agile coaches demand it’s reasonable to expect these people to be qualified.

Our Solution: Agile Coaching Focused Certifications

A fair question to ask is how do we deal with this in the Disciplined Agile (DA) space. We believe that it’s critical to your success to have qualified coaches so we’ve built a principled certification program based on the martial arts philosophy of Shu-Ha-Ri.  Certifications must be earned and that takes time.  PMI offers three certifications that teach Agile coaching skills and knowledge:

1. Disciplined Agile Senior Scrum Master (DASSM). This certification addresses fundamental people and collaboration skills that are critical for anyone in a senior team position, including coaches, team leads, and architects.  More importantly it teaches how to lead agile teams in their process improvement efforts, including both software and non-software teams. Anyone hoping to become an agile team coach or team improvement coach should be interested in this certification. DASSM should be particularly attractive to anyone who is a Scrum or SAFe Coach, which have become commodities these days, as it teaches you how to lead improvements that go far beyond what you learn as a method-specific coach.
2. Disciplined Agile Coach (DAC). This certification is aimed at anyone wanting to become a senior agile coach.  DAC focuses on how to coach across disparate teams. What we mean by this is that we teach how to approach a problem where the team you are currently coaching needs to negotiate a new WoW with another team that is very different than their own. For example, how can you help your team work more effectively with your organization's procurement team, which has a different outlook, priorities, and WoW than your team?  DAC also teams how to coach non-software teams.
3. Disciplined Agile Value Stream Consultant (DAVSC). This certification focuses on how to define and evolve value streams within your organization.  Value streams are how you provide products and services to your customers. This certification is aimed at people wanting to become enterprise agile coaches or value stream coaches.

Parting Thoughts

It isn’t easy to find qualified agile coaches, but then again it isn’t impossible either. Our hope is that this blog has provided you with some insight into what you should be looking for in a good agile coach.  Anyone can put a shingle up and say that they’re an “agile coach”, but anyone who wants to say that they are a Disciplined Agile certified coach needs to have worked through a rigorous process to earn that qualification.  DASSMs, DACs, and DAVSCs have proven knowledge, experience, and give back.  Why settle for less?

Related Reading

• Disciplined Agile Certification
• The Principles Behind Disciplined Agile Certification
• The Limits of Certification

There are several values that are key to your success when transforming to a leaner, more agile approach to Data Management. Taking a cue from the Disciplined Agile Manifesto, we’ve captured these values in the form of X over Y. While both X and Y are important, X proves to be far more important than Y in practice. These values are:

1. Evolution over definition. The ability to safely and quickly evolve an existing data source, either to extend it to support new functionality or to fix quality issues with it, is absolutely paramount in today’s hyper-competitive environment. Yes, having defined data models and metadata describing them is also important, but nowhere near as important as being able to react to new business opportunities. Luckily agile database techniques, long proven in practice, exist that enable the safe evolution of production data stores.
2. Holistic organization over Data Management. Earlier we said that data is the lifeblood of your organization. Yes, blood is important but so is your skeleton, your muscles, your organs, and many other body parts. We need to optimize the whole organizational body, not just the “data blood.” Traditional Data Management approaches often run aground because they locally optimize for data concerns, whereas a DA approach to Data Management recognizes that we must optimize the overall whole. This implies that sometimes we may need to sub-optimize our strategy from a data point of view, for the sake of organizational level optimization.
3. Sufficiency over perfection. Data sources, like many other IT assets, need to be good enough for the task at hand. The old saw “perfect is the enemy of good” clearly applies in the data realm – too much time has been lost, and opportunities squandered, while development teams were forced to wait on Data Management teams to create (near) perfect models before being allowed to move forward. Traditional data professionals mistakenly assume that production databases are difficult to evolve and as a result strive to get their designs right the first time so as to avoid very painful database changes in the future. The Agile Data method has of course shown this assumption to be wrong, that it is very straightforward and desirable to evolve production databases. A side effect of this revelation is that we no longer need to strive for perfect, detailed models up front. Instead we can do just enough up-front thinking to get going in the right direction and then evolve our implementation (including data sources) over time as our understanding of our stakeholder needs evolve.
4. Collaboration over documentation. We’ve known for decades that the most effective way to communicate information is face-to-face around a shared sketching environment, and that the least effective is to provide detailed documentation to people. The implication is that we need to refocus our efforts to be more collaborative in nature. As data professionals we need to get actively involved with solution delivery teams: to share our knowledge and skills with those teams, and to enable them to become more effective in working with data. Yes, we will still need to develop and sustain data-related artifacts, but those artifacts should be lightweight and better yet executable in nature.
5. Cross-functional people over specialized staff. Agilists have come to realize that people are more effective when they are cross-functional (also known as T-skilled or generalizing specialists). Although specialists are very skilled in a narrow aspect of the overall process, the problem is that you need a lot of specialists to perform anything of value and as a result the overall workflow tends to be error prone, slow, and expensive. The other extreme would be to be a generalist, someone who knows a little bit about all aspects of the overall process. But, the challenge with these people is that although they’re good at collaborating with others they don’t actually have the skills to produce concrete value. We need the best of both worlds – a generalizing specialist with one or more specialties so that they can add value AND a general knowledge so that they can collaborate effectively with others and streamline the overall effort.
6. Automation over manual processes. The only way that we can respond quickly to marketplace opportunities is to automate as much of the bureaucracy as we possibly can. Instead of creating detailed models and documents and then reviewing potential changes against them we capture our detailed specifications in the form of executable tests. This is quickly becoming the norm for specifying both the requirements and designs of code, and the same test-driven techniques are now being applied to data sources. Continuous integration (CI) and continuous deployment (CD) are also being applied to data sources, contributing to improving overall data quality and decreasing the time to safely deploy database updates into production.

As you can see, we’re not talking about your grandfather’s approach to Data Management. Organizations are now shifting from the slow and documentation-heavy bureaucratic strategies of traditional Data Management towards the collaborative, streamlined, and quality-driven agile/lean strategies that focus on enabling others rather than controlling them.

Recommended Reading

• To read more, we have a detailed article overviewing a Disciplined Agile approach to Data Management
• Need training in agile skills for DW/BI?  Consider the workshop Disciplined Agile Data Warehousing(DW)/Business Intelligence (BI).