There are several values that are key to your success when transforming to a leaner, more agile approach to Data Management. Taking a cue from the Disciplined Agile Manifesto, we’ve captured these values in the form of X over Y. While both X and Y are important, X proves to be far more important than Y in practice. These values are:
As you can see, we’re not talking about your grandfather’s approach to Data Management. Organizations are now shifting from the slow and documentation-heavy bureaucratic strategies of traditional Data Management towards the collaborative, streamlined, and quality-driven agile/lean strategies that focus on enabling others rather than controlling them.
For teams that are applying agile strategies to Data Warehouse (DW)/Business Intelligence (BI) development is it fairly common for them to take a Disciplined Agile (DA) Approach to DW/BI due to DA’s robustness. A common question that comes up is how do you write user stories for DW/BI solutions? Here are our experiences.
First, user stories should focus on business value. In general, your stories should answer the question “What value is being provided to the user of this solution?” In the case of a DW/BI solution, they should identify what question the DW/BI solution could help someone to answer, or what business decision the solution could support. So “As a Bank Manager I would like the Customer Summary Report” or “Get Customer data from CustDB17 and TradeDB” would both be poorly written user stories because they’re not focused on business value. However, “As a Bank Manager I would like to know what services a given customer currently have with BigBankCo so that I can identify what to upsell them” is. The solution to implement that story may require you to create the Customer Summary Report (or there may be better ways at getting at that information) and it may require you to get data from CustDB17 and TradeDB (and other sources perhaps).
Here are some examples of user stories for a University DW/BI solution:
Second, user stories on their own aren’t sufficient. User stories/epics are only one view into your requirements, albeit an important one. You’ll also want to explore the domain (e.g. do some data modelling), the user interface (e.g. explore what reports should look like), the business process (e.g. what are the overall business process(es) supported by your DW/BI solution), and technical views (e.g. how does the data flow through your solution architecture).
Third, data requirements are best addressed by domain modelling. As you are exploring the requirements for your DW/BI solution you will hear about data-oriented requirements. So capture them in your domain model as those sorts of details emerge over time. Consider reading Agile/Evolutionary Data Modeling: From Domain Modeling to Physical Data Modeling for a detailed discussion of this topic.
Fourth, technology issues are best captured in architecture models. You will also hear about existing legacy data sources and in general you will need to capture the architecture of your DW/BI solution. This sort of information is best captured in architecture models, not in stories.
A few more thoughts:
This posting, the latest in a series focused on a disciplined agile approach to data management, overviews the activities that a disciplined agile data management team may perform. The Disciplined Agile (DA) toolkit promotes an adaptive, context-sensitive strategy to data management. DAD does this via its goal-driven approach that indicates the process factors you need to consider, a range of techniques or strategies for you to address each process factor, and the advantages and disadvantages of each technique. In this blog we present the goal diagram for the Data Management process blade and overview its process factors.
The following diagram overviews the potential activities associated with disciplined agile data management.
The process factors that you need to consider for data management are:
Looking at the diagram above, traditional data management professionals may believe that some activities are missing. These activities may include:
Future blog postings in this series will explore the workflow associated with data management.
According to the Data Management Body of Knowledge, data management is “the development, execution and supervision of plans, policies, programs and practices that control, protect, deliver and enhance the value of data and information assets.” In our opinion this is a very good definition, unfortunately the implementation of data management strategies tends to be challenged in practice due to the traditional, documentation-heavy mindset. This mindset tends to result in onerous, bureaucratic strategies that more often than not struggle to support the goals of your organization.
Having said that, data management is still very important to the success of your organization. The Disciplined Agile (DA) toolkit promotes a pragmatic, streamlined approach to data management that fits into the rest of your IT processes – we need to optimize the entire workflow, not sub-optimize our data management strategy. We need to support the overall needs of our organization, producing real value for our stakeholders. Disciplined agile data management does this in an evolutionary and collaborative manner, via concrete data management strategies that provide the right data at the right time to the right people.
There are several reasons why a disciplined agile approach data management is important:
In future blog postings we will explore the goal diagram of the Data Management process blade and the associated workflow.