Project Management

Disciplined Agile

by , , , , , ,
#ChooseYourWoW | #ContinuousImprovement | #Kaizen | #ProcessImprovement | Adoption | agile | Agile certification | agile transformation | Analogy | Architecture | architecture | book | Business Agility | Certification | Choose your WoW | CMMI | Coaching | Collaboration | Compliancy | Configuration management | Construction phase | Context | Continuous Improvement | COVID-19 | Culture | culture | DAD | DAD discussions | DAD roles | Data Management | database | DevOps | Discipline | disciplined agile delivery | Documentation | DW/BI | Enterprise Agile | Enterprise Architecture | Enterprise Awareness | Essence | Evolving DA | Experiment | Financial | GDD | Geographic Distribution | global development | Goal-Driven | goal-driven | goals | Governance | Guideline | Hybrid | Improvement | inception | Inception phase | Kanban | Large Teams | Lean | Lifecycle | lifecycle | Metrics | mindset | News | News and events | Non-Functional Requirements | non-functional requirements | Operations | Outsourcing | People | Philosophies | Planning | PMI | PMI and DA | Portfolio Management | Practices | Principle | Process | process improvement | Product Management | Product Owners | Program Management | Project Management | Promise | quality | Release Management | Requirements | requirements | Reuse Engineering | Risk management | RUP | Scaling | scaling | scaling agile | Scrum | Support | Surveys | Teams | Technical Debt | Terminology | Testing | testing | Toolkit | Transformation | Workflow | show all posts

About this Blog

RSS

View Posts By:

Scott Ambler
Glen Little
Mark Lines
Valentin Mocanu
Daniel Gagnon
Michael Richardson
Joshua Barnes

Recent Posts

Failure Bow: Choosing Between Life Cycles Flowchart Update

Evolving Disciplined Agile: Guidelines of the DA Mindset

Evolving Disciplined Agile: Promises of the DA Mindset

Evolving Disciplined Agile: Principles of the DA Mindset

Evolving Disciplined Agile: The DA Mindset

Bridging the Gap Between Agile and Data


At the Agile 2017 conference this week Lynn Winterboer was kind enough to invite me to her workshop which explored how to apply agile strategies in the data space.  She did a great job of facilitating the group of about 40 people through identifying the challenges currently faced by teams. The main issues that the group explored were:

  • Product ownership and data
  • Agile data architecture
  • Data testing and tooling support
  • How to include data people, and activities, in development

Lynn will soon be blogging about the results so I’m not going to dive into that here.  I suspect that her blog post will be very interesting.

What I’d like to do here is share a few thoughts about what I observed:

  1. The discussion was very healthy. This was a group of very smart people coming from different backgrounds.  Everyone was interested in sharing their experiences and working together to solve some tough problems.
  2. Context counts. “Agile and data” is a big topic.  A few people were dealing with the issue of how to address data issues on application development projects, some were focused on agile data warehousing/business intelligence, and some were focused on complex data analytics.  In our conversations it was very clear that strategies which worked for app development wouldn’t work for analytics, and vice versa.  This is why Context Counts is one of Disciplined Agile’s fundamental principles.
  3. The challenges are tough. Everyone was working in organizations that were struggling with these challenges.  For each of issue we could have spent much longer exploring the potential solution(s).
  4. Every challenge we discussed are solved issues.  I’ve always found it frustrating when people, who are very smart and who have been struggling with a problem for awhile, have clearly never thought to simply google “database testing tools” or “agile architecture” to find out what advice is already out there.  When we discussed testing I even asked why people hadn’t done such as search, and then pointed out that there are a lot of tools available and even a few people maintaining lists of such tools (such as 40+ database testing tools).  All of these challenges, and more, have solutions described by techniques of the Agile Data and Agile Modeling methods from about 15 years ago.  These techniques have of course been adopted, and put into context, by the Disciplined Agile (DA) toolkit and in particular Disciplined Agile Delivery (DAD).
  5. The “factions” behaved differently.  Long ago I pointed out that there’s a cultural impedance mismatch between the data and developer communities, and it was pretty easy to observe during this workshop.  For example, during the workshop we were asked to identify solutions to the challenges.  Lynn wanted us to write this information on flip-chart paper so that she could later scan it and share it with others.  For awhile the groups dominated by data people discussed the solutions without writing anything down.  Their conversations were good, but they quickly got stuck in analysis paralysis mode because they seemed to be afraid to write down information for fear that they couldn’t easily update it (the challenge with having paper to write on instead of whiteboards). The groups dominated by agile people, the ones focused on development and architecture, started by writing on sticky notes and putting them on the flip chart paper.  This fundamental agile modelling strategy enabled them to visualize and share their work transparently, enhancing their conversation and enabling them to move forward easily.
  6. Database (tool) vendors need to up their game. Speaking out tools, database vendors and data warehouse tool vendors really need to up their game.  Here’s a few very harsh questions: Does your database tool vendor or ETL tool vendor offer testing tools that enable both black box and white box database testing?  Answer: very likely no, because their customers don’t demand that of them.  Did your data team even think to ask for such tools?  Answer: very likely not.  How many database testing books do you think have been written? Answer: very few, go do a search and see for yourself.  Why does the data community have such a huge blind spot when it comes to testing? Answer: this is a huge side effect of the cultural impedance mismatch.

I’m very happy to see that Lynn is actively trying to bridge the agile and data communities, helping us to learn from each other.  This is something she’s been doing for years and doing it quite well.  My experience is that both communities would benefit greatly from more collaboration with each other.

Posted by Scott Ambler on: August 11, 2017 08:37 AM | Permalink | Comments (0)

Vertical Slicing for a Data Warehouse (DW)/Business Intelligence (BI) Solution

Categories: database, DW/BI

There are several strategies that you can choose to employ with vertically slicing the requirements for a DW/BI solution. These strategies are described in the following table. There are example stories for each strategy as well as some advice for when to apply each strategy.

Table 1. Vertical slicing strategies for a DW/BI solution.

Slicing Strategy Example Stories When to Do This
One new data element from a single data source
  • As a Professor I would like to know the names of my students so that I know who should be there
  • As a Student I would like to know what courses are taught at the university
Very early days when you are still building out fundamental infrastructure components. Very common for the first iteration or two of Construction. These slices still add real business value, albeit minimal.
One new data element from several sources
  • As a Professor I would like the student list for a seminar that I teach
  • As a Student I would like to know what seminars are being taught this semester
Early days during Construction when you are still building out the infrastructure. These slices add some business value, often fleshing a DW data element to include the full range of data values for it.
A change to an existing report
  • As a Professor I would like to know the standard deviation of marks within a seminar that I teach
  • As a Student I would like to know how many spots are still available in a seminar
Evolution of existing functionality to support new decision making
A new report
  • As a Professor I would like to know the distribution curve of student marks in a seminar that I teach so I may adjust accordingly
  • As a Registrar I would like to know what Seminars are close to being full
Several iterations into Construction when the DW/BI solution has been built up sufficiently.
A new reporting view
  • As a Registrar I would like to know what the prerequisites are for a seminar so that I can advise students
  • As a Professor I would like to know the current course load of each student within a seminar that I teach
Several iterations into Construction when the DW/BI solution has been built up sufficiently.
A new DW/DM table
  • As a Chancellor I would like to track the revenues generated from parking pay meters to identify potential profits to divert to supporting students
  • As a Professor I would like to recommend suggested readings to help people prepare before taking a seminar
Several iterations into Construction when the DW/BI solution has been built up sufficiently.

There are several interesting things about the stories in the table:

  1. They are written from the point of view of your stakeholders. They aren’t a technical specification. For example, the first story describes how professors want a list of student names but it isn’t saying from what data source(s), what the element names are, … These are design issues, not requirement issues.
  2. They always provide business value. The first story appears to be the beginnings of an attendee list for a seminar. Having something as simple as a list of names does in fact provide a bit of value to professors.
  3. Sometimes that business value isn’t (yet) sufficient. It may take several iterations to implement something that your stakeholders want delivered into production, particularly at first. For example, although a list of student names is the beginnings of a class list it might not be enough functionality to justify putting it into production. Perhaps professors also need to know the program that the student is enrolled in, their current year of study, and basic information about the seminar such as the course name, time, and location of it. The decision as to whether the functionality is sufficient to ship is in the hands of your stakeholder (this is one of the reasons why you want to demo your work on a regular basis).

I’ve written a detailed explanation of vertical slicing for a DW/BI solution, and of course there is a wealth of information about agile database techniques in general for those of you interested in greater detail.  You may also find our one-day Disciplined Agile Data Warehousing/Business Intelligence Workshop to be a valuable learning experience too.

Posted by Scott Ambler on: February 06, 2017 03:06 PM | Permalink | Comments (0)
ADVERTISEMENTS

"Of course the music is a great difficulty. You see, if one plays good music, people don't listen, and if one plays bad music, people don't talk."

- Oscar Wilde

ADVERTISEMENT

Sponsors