How Do You Coach an Agile DW/BI Team?

 

At the Agile 2019 conference in DC I facilitated a workshop with about 70 people where we explored the topic of how do you coach an agile data warehousing (DW)/business intelligence (BI) team. To do this we worked through four issues:

1. What challenges do you face?
2. What skills/knowledge does an agile DW/BI coach require?
3. What strategies should you apply to engage with a DW/BI team?
4. What are the qualities you should look for in an agile DW/BI coach?

The basic strategy was to introduce the issues to the class one at a time, then at their tables they would discuss the issue and write up to five ideas on sticky notes, then we’d share the ideas. Pictures of the flipcharts for each issue follow below. After the groups shared their ideas I then shared my thoughts with the class.

Issue #1: What Challenges Do You Face Coaching DW/BI Teams?

As you can see the class identified a lot of the classic issues that agile coaches face in general, such as trust issues, the teams being management-driven instead of self organizing, lack of agile skills within the team, cross-team dependencies, and being overwhelmed with work. Certainly there were DW/BI flavours of common problems, such as how to do vertical slices of DW/BI functionality and which lifecycle (agile, lean, CD, …) to follow. But there were also DW/BI specific issues, such as lack of access to data sources, knowing the actual data, and DW/BI architecture and design strategies. These DW/BI specific issues are where agile coaches tend to get hung up.

In my discussion of the challenges that we face when coaching agile DW/BI teams I shared my thoughts on the cultural impedance mismatch that exists between the agile and data communities. This mismatch makes it a bit more difficult to engage with data teams as opposed to application development teams. I also shared results of studies (2009, 2013,2016, 2018) around data quality challenges and practices – it is certainly common for teams to have to deal with technical debt, but data technical debt is both different in nature than code quality debt and the traditional data culture has led them down a very questionable (read dysfunctional) path regarding quality practices.

Issue #2: What Skills/Knowledge Does an Agile DW/BI Coach Require?

The second issue that we explored was what skills/knowledge does an agile DW/BI coach need. Once again the groups identified both classic agile coaching ideas as well as DW/BI specific ideas. Clearly you need coaching skills in order to coach a DW/BI team. But you also need to be knowledgeable about critical skills such as data modeling, data analysis, database testing, database refactoring, and others. You might not be an expert at these things, but you need to know of them and be able to guide the team in their adoption. You’ll also need to be able to speak intelligently about why some of the traditional strategies that they likely hold near and dear to their hearts (remember the cultural impedance mismatch) need to be abandoned for better, more effective strategies.

In my discussion I overviewed the “agile database techniques stack,” a collection of agile strategies and practices for database development. The stack is:

• Vertical slicing
• Clean data architecture
• Clean database design
• Agile data modeling
• Database refactoring
• Database regression testing
• Continuous database integration
• Database configuration management

As you can see, this list of techniques is fairly common from an agile point of view, albeit the corresponding data(base) versions of those techniques. The point is that the techniques exist that enable data professionals to work in an agile, and far more effective, manner. As a coach you will need to be aware of these strategies and be able to help your DW/BI team adopt them. And of course there are agile data management strategies that you need to be aware of as well.

Issue #3: What Strategies Should You Use To Engage Successfully With An Agile DW/BI Team?

The groups identified a collection of great strategies for engaging with DW/BI teams. Once again there were a lot of standard coaching strategies, a DW/BI team is still a group of people after all, but there was also a focus on strategies to address the DW/BI challenges identified earlier.

The discussion that followed the sharing of the stickies a very interesting point was brought up. I had earlier stated that my experience with coaching DW/BI teams was that it was different than coaching other types of teams, mostly because of the cultural impedance mismatch. A handful of agile DW/BI coaches in the audience disagreed with that, pointing out that the critical issue was gaining the trust and respect of the team at the start. This is true of any team, and certainly true of DW/BI teams. To do this you need to understand and appreciate the issues that they deal with and be able to show that you know how to guide them through addressing their issues. You might not be an expert in the techniques of the agile database technique stack, or other important agile data techniques, but you do know of them and can help the team learn them. So yes, engaging with an agile DW/BI team is no different on the surface, but it does require the coach to have different skills and knowledge than what your typical agile coach has.

Issues #4: What Are The Qualities You Should Look For In An Agile DW/BI Coach?

For this exercise I pretty much asked the groups to put together a list of qualities for a job ad for an Agile DW/BI coach. This is what they came up with.

Our Learnings

Here are our main learnings regarding coaching an agile DW/BI team:

• You need fundamental agile coaching skills
• You need to at least be aware of, but better yet experienced with, agile data and agile database development strategies
• To effectively engage with a DW/BI team, you need to gain their trust and respect – to do that you need to show that you understand the challenges that they face and can help them to address them
• To do so can be more challenging than with an application team due to the cultural impedance mismatch, the nature of data-oriented technical debt, and the lack of modern development skills within the data community – all of these challenges point to a greater than normal need for coaching on such teams