Disciplined Agile

As with most things in IT, there is no standard definition of what IT governance is. For example, the IT Governance Institute provides this definition:

“IT Governance is the leadership, organizational structures and processes to ensure that the organization’s IT sustains and extends the organization’s strategies and objectives.”

Gartner, on the other hand, offers this definition:

“IT governance (ITG) is defined as the processes that ensure the effective and efficient use of IT in enabling an organization to achieve its goals. IT demand governance (ITDG) is the process by which organizations ensure the effective evaluation, selection, prioritization, and funding of competing IT investments; oversee their implementation; and extract (measurable) business benefits. ITDG is a business investment decision-making and oversight process, and it is a business management responsibility. IT supply-side governance (ITSG) is concerned with ensuring that the IT organization operates in an effective, efficient and compliant fashion, and it is primarily a CIO responsibility.”

In Disciplined Agile, we promote the following definition:

“Lean IT Governance is the leadership, organizational structures and streamlined processes to enable IT to work as a partner in sustaining and extending the organization’s ability to produce meaningful value for its customers.”

As you can see in the following diagram, IT governance is part of your organization’s overall corporate governance strategy. IT governance encompasses several more narrow forms of governance, including but not limited to the governance of IT delivery/development, data/information, IT security, IT investment, enterprise architecture, and IT operations activities.

IT governance typically addresses areas such as:

• The effective and timely investment in IT to sustain and extend the organization over the long term
• The evolution and support of roles and responsibilities to streamline how people work together
• Definition of decision rights and decision making processes to streamline interactions between people
• The evolution and support of common procedures and guidelines to ensure appropriate commonality of activities and artifacts
• The evolution and support common roadmaps to guide the efforts of IT teams
• The monitoring of activities to provide insight into their effectiveness
• Formation of a governing body that is responsible for guiding governance activities
• Definition of exceptions and escalation processes to streamline critical interactions
• Creation of a knowledge sharing strategy to grow individuals, teams, and the organization as a whole
• The support and monitoring of risk mitigation strategies to promote appropriate and holistic adoption of IT solutions
• Adoption of a reward and compensation structure to support the attraction and retention of excellent staff
• Status reporting to share information throughout the organization

In future blog postings we will explore why IT governance is important, the differences between traditional and lean approaches to IT governance, and the activities addressed by IT governance.

Related Reading:

• What is Agile Delivery Governance?
• Strategies that Enable Agile Governance
• Supporting Agile Delivery Team Governance

What is the point of the retrospective?

The retrospective is one of the most important ceremonies in all of agile.  This is the time the team spends together to assess how they are working together and define steps to improve that process.  It needs to be a “safe place” where people are able to speak openly and honestly.  This is their opportunity to air their dirty laundry and work through their inter-personal issues.  This is a time of growth for the team and for the team to take ownership of improvement.  The team lead will facilitating the retrospective and should manage the interactions to keep the environment safe.

Define the Team

If the retrospective is a team ceremony, then what do we mean by team?   The team includes: the team lead, the architecture owner and all other members that actively contributed to meeting the deliverables for the iteration.  This includes: developers, testers, BAs, QAs, or other specialists such as technical writers, database engineers etc.

What about the Product Owner (PO)?

The PO is NOT a member of the team.  They certainly interact with the team but they do not contribute to meeting the deliverables for the iteration so they are not a member of the team.  They are not allowed to participate in the planning poker for the user stories for the same reason.  The team votes because they are on the hook for delivering based on the sizing and the estimates but the PO is not on the hook, so they don’t get a vote.

Should the Product Owner participate in the retrospective?

In general, I would start by including the PO in the retrospectives because the team does have to learn and adjust to working with the PO. Keep in mind though that the PO may come and go but the team should stay together so it is most important that the team works well together.??As a coach, I usually talk to the PO beforehand to say that they are an invited guest and that it is a privilege to be part of the meeting so they should act accordingly.   I have been in many situations where the PO was welcomed at the retrospective, and felt left out if not included.  I favor building trust between the business (the PO is their representative) and IT (the team).  Including the PO in the retrospective can help the PO assimilate with the team.

I have also had several situations where as the coach I had to ban the PO from the retrospective because they were too commanding and disruptive in the meeting for the team to have an effective retrospective.  I have also seen many situations where the PO is also the resource manager of members of the team (which in itself is not recommended).  Having managers in the room can definitely have a dampening effect on the member’s willingness to be open and honest about problems and solutions.

If the PO doesn’t participate, at least as an observer, the team runs the risk of having to “sell” the cost of their improvement actions (against other backlog items) after coming up with them. Hopefully the PO is engaged enough with the team to understand its weaknesses and support improvement in those areas whether then attend the meetings or not.

Team Decision

Retrospectives are about improving the process, and a non-trivial part of that is optimization of collaboration between the PO and the team.  I would suggest that the team should decide whether or not to include the Product owner.

What about the Stakeholders?

The retrospective should absolutely be a closed door session for the stakeholders since the retrospective must be a “safe space”.

There was a twitter debate lately that talked about a team being subjected to “a drive by criticism from 2 PM’s during a Retrospective”.  This is a good reminder why we constrain attendance.  The “safe place” is affected by the presence of people with positional authority, potential agendas or other implicit impact.??The team may decide to invite such people – usually to ensure that they are communicating improvements needed that are beyond their locus of control.??Having outsiders as guests at the retrospective will change the dynamics but at least it is a team decision to do so.

It is very important that the team own their process.  If they’re uncomfortable that someone is in the room then that person should be asked to either change their behavior or leave (perhaps to be invited back in the future).??The coach should always be thinking along the lines of “do we have the right people in the room” and then act accordingly

Isn’t agile all about transparency?

There was a twitter debate lately that centered on transparency.  I believe that transparency is a key element to making agile successful.   I’m all for transparency in everything about agile; EXCEPT the retrospective!??Sometimes you need to have a family meeting outside of the public eye and that is the retrospective. The retrospective is all about resolving your issues in private so that you can present a united front to the rest of the world. To use a sports analogy, an NHL coach doesn’t invite the business (fans) into the dressing room between periods.  There are lots of other places for transparency; the retrospective doesn’t have to be one of them.

The output of the Retrospective

While the actual retrospective meeting is closed to other observers, I would suggest that the action items coming out of the retrospective need to be made public and posted as an information radiator for everyone to see.  The changes are more likely to get implemented if the team sees them every day.  The team may also want to “radiate” their improvement actions on their dashboard.

The actions and results of the actions may also be shared with other teams through what is often called a Retrospective of Retrospectives. I encourage teams to only choose one or two areas for improvement at a time to provide focus and make meaningful progress.

For teams that are applying agile strategies to Data Warehouse (DW)/Business Intelligence (BI) development is it fairly common for them to take a Disciplined Agile (DA) Approach to DW/BI due to DA’s robustness.  A common question that comes up is how do you write user stories for DW/BI solutions?  Here are our experiences.

First, user stories should focus on business value.  In general, your stories should answer the question “What value is being provided to the user of this solution?”  In the case of a DW/BI solution, they should identify what question the DW/BI solution could help someone to answer, or what business decision the solution could support.  So “As a Bank Manager I would like the Customer Summary Report” or “Get Customer data from CustDB17 and TradeDB” would both be poorly written user stories because they’re not focused on business value.  However, “As a Bank Manager I would like to know what services a given customer currently have with BigBankCo so that I can identify what to upsell them” is.  The solution to implement that story may require you to create the Customer Summary Report (or there may be better ways at getting at that information) and it may require you to get data from CustDB17 and TradeDB (and other sources perhaps).

Here are some examples of user stories for a University DW/BI solution:

• As a Professor I would like to analyze the current grades of my students so that I can adjust the difficulty of future tests and assignments
• As a Student I would like to know the drop out rates by course and professor from previous years to determine the likely difficulty of my course choices
• As a Registrar I would like to know the rate of enrollments within a class over time to determine the popularity of them
• As a Student I would like to know the estimated travel time between back-to-back classes so that I can determine whether I can make it to class on time

Second, user stories on their own aren’t sufficient.  User stories/epics are only one view into your requirements, albeit an important one.  You’ll also want to explore the domain (e.g. do some data modelling), the user interface (e.g. explore what reports should look like), the business process (e.g. what are the overall business process(es) supported by your DW/BI solution), and technical views (e.g. how does the data flow through your solution architecture).

Third, data requirements are best addressed by domain modelling.  As you are exploring the requirements for your DW/BI solution you will hear about data-oriented requirements.  So capture them in your domain model as those sorts of details emerge over time.  Consider reading Agile/Evolutionary Data Modeling: From Domain Modeling to Physical Data Modeling for a detailed discussion of this topic.

Fourth, technology issues are best captured in architecture models.  You will also hear about existing legacy data sources and in general you will need to capture the architecture of your DW/BI solution.  This sort of information is best captured in architecture models, not in stories.

A few more thoughts:

1. User stories are only one option for usage modelling.  There are several ways that we can explore usage, user stories/epics are just one way.  You could also create light-weight use cases, usage scenarios, or personas to name a few strategies.
2. Take a usage-driven approach.  The primary modelling artifact on a Disciplined Agile DW/BI team is the usage model (e.g. your user stories) not the data model.  Data modelling is a secondary consideration compared with usage modelling, and this can be a difficult concept for experienced DW/BI professionals to come to terms with.
3. Keep your initial modelling light-weight.  The agile rules still apply to DW/BI solutions – keep your modelling efforts sufficient for the task at hand and no more.
4. Get trained in this.  This is a complex topic.  

Over the past few days there has been a Tweetstorm around Agilia Conference 2016 that is being held April 4-8 in Olomouc, Czech Republic.  I’ve decided to call this Agiliagate as every “scandal” these days seems to named in honour of the Watergate scandal that brought down Richard Nixon in the 1970s. The scandal started with a tweet from Samantha (Sam) Laing.  For those of you who don’t know her, Sam is an agile coach based out of Cape Town South Africa.  She has co-authored several excellent books and is a regular speaker at conferences worldwide.  I’ve known and respected her for several years now and was lucky enough to take a coaching workshop with her a few years ago at the Agile conference in the U.S.  Her tweet was:

wow @agiliaconf really? 2 out of 27 speakers are woman? You couldn’t find any more?

Then Things Spun Out of Control

Sam made a perfectly valid observation.  Sadly, the initial response to Sam’s question was very, very unfortunate.  @agiliaconf responded with:

@samlaing Why we shoud worry? We do not play feminist games here nor politics. Whoever pass our extensive screenening, he may come.

Yikes.  Just yikes.  Sam then maturely responded:

@agiliaconf not a feminist game, just shocked at lack of diversity and now, lack of caring about diversity. It’s ok, just my opinion.

A very fair and level-headed response in my opinion.  Digging their hole even deeper, @agiliaconf responded with:

@samlaing Yes, as I said, we are not political party. Go to EU in Bruxelles, they love to engineer diversity games. Our focus is different.

That response clearly didn’t help.  By this point the Tweetstorm was in full swing, although @agiliaconf chose to make matters even worse by tweeting:

You have enemies? Good. It means you’ve stood up for something, sometime in your life.- Sir Winston Churchill – short rep to todays attack

@Agiliaconf’s first tweet could be chalked up as an unfortunate mistake.  Their second one was highly questionable, but the third one spectacularly stupid.  Dozens of people from all over the world, many of them important contributors in the agile community, were actively criticizing the conference organizers for the lack of gender diversity at the event.  Some people were also choosing to try to publicly shame several of the speakers, including myself, for being involved with the conference.  Some people tried to bully speakers to drop out of participating in the conference.  Others started going after the vendors who were sponsoring the conference.  Villains had been clearly identified and the forces of righteous indignation were out for blood.

Out of the Mess, Some Very Good Suggestions

The good news is that during all of this several good suggestions came out of the cacophony.  These suggestions included:

1. Invite more female speakers.  Clearly a great idea for future events, but considering this conference was less than a week away it wasn’t a viable consideration to address the immediate problem.
2. People should only speak at conferences that support diversity.  That’s also a great idea, and I’ll going start asking about this in the future, so lesson learned for me.
3. Invite a woman to co-present at the conference.  This is also a great idea.  I’ve co-presented with many people in the past and that works very well and is a great way to shepherd people to become public speakers.
4. Invite one or more women to be involved with the conference committee.  This is a great idea for future events.  Having been involved with many conference committees over the years my experience is that the more diverse your conference committee then generally the more diverse, and interesting, your speaker line-up will be.

Let’s Add Some Context

As we like to say at my company, context counts.  Had the people involved taken a few minutes to pause and consider the situation, I suspect they would have identified some important contextual factors:

1. This happened over a low-fidelity communication medium.  Communication via electronic text, and in particularly the 140-character messages of Twitter, is problematic at best (see my article Agile Communication for some detailed thoughts on this topic).  It is very difficult to have meaningful conversations via Twitter, and it takes people to be a bit more careful in the way that they word things and thoughtful in how they react.  This clearly wasn’t happening.
2. Organizing a conference is a lot of hard, stressful work.  It’s particularly stressful in the days leading up the the conference, which is the period when all of this happened.  @Agiliaconf was very likely dealing with a fair bit of stress at the time that Sam’s tweet came in.  Some of the people critical of @Agiliaconf may not have much, if any experience, with how conferences are run so I can see them being unaware of this.  However, several of the detractors are regular conference speakers and should have been a lot more empathetic.
3. There are cultural differences.  Aguarra, the organization running the conference, is based in the Czech Republic.  The Czechs have their own unique culture, just like the Canadians do, the Brits, the Germans, the South Africans, and so on.  There are some great similarities between all of these cultures, but important nuances too.  When people from different cultures are interacting with one another it often requires a bit more patience and latitude.

The point is that instead of responding with a knee-jerk reaction I decided to act in a respectful manner and contact the Agilia folks and see what they had to say for themselves.

Agilia’s Side of the Story

I had a email conversation with Michal, one of the two organizers of event.  Here’s a few things that I learned in the conversation:

1. Their native language isn’t English.  In hindsight this is clear given the wording of some of their tweets.  It’s hard enough to have a conversation on Twitter, but doing so in a language that isn’t your first one is very difficult.
2. They’re fairly new to Twitter.  Since joining Twitter they’ve sent less than 250 tweets, many of which are focused on advertising presentations at conferences.  As you can see from their tweets they perceived Sam’s initial tweet as an attack coming from a complete stranger, and with just a bit of empathy I think it’s pretty easy to see how this would be the case.  Furthermore, they honestly thought that their first response was sarcastic and would be taken as such.  I explained that sarcasm is an incredibly bad idea on Twitter at the best of times, and that they really needed to avoid it in the future.
3. Half of the conference organizers are female.  Granted, there are only two conference organizers, a man and a woman.  As you can see from the link, this information is fairly easy to discover yet none of their critics bothered to look into this (or if they did they certainly didn’t tweet about it).
4. The critics are from outside of Central and Eastern Europe (CEE).  The conference organizers did some analysis of where the critical tweets were coming from, and for the most part it was from outside of the CEE. This is a reflection of my earlier point about cultural differences between the people involved.  When this happens everyone needs to be more patient and understanding.
5. The critics didn’t bother to reach out to the Agilia organizers.  Because I didn’t know what was going on behind the scenes, I asked if any of the critics had bothered to reach out to the Agilia organizers to get their side of the story.  Here was the response: “From speakers, only Ben Linders and Olli Pietikainen. From people, who posted some articles about this situation, nobody. From most visible twitter screamers such as [NAMES WITHHELD BY ME] nobody. From other twitter people – nobody.”  Given that the agile community preaches values such as respect, communication, and collaboration many of our more prominent people have clearly failed to fulfill these ideals in the “AgiliaGate” situation.  I am embarrassed for us.
6. Their responses reflect their culture.  When I pointed out to them that their responses to Sam were inappropriate, something that many others pointed out via Twitter, here was their response: “I will appreciate, if I could get more detailed info of what has happened as seen from other side. In our cultural environment, there is general dissagreement in society about attempts to impose quotas on anything, including number of women or minorities or anything.  Asking for it might be interpreted as an assalt, which I did.”  Yes, many of us may not agree with that but this is the cultural environment in which the Agilia conference is operating.  Asked about what they were thinking with their second tweet, they responded “I responded with lite sarcasm, true, that we praise meritocracy here.  In second tweet the my message was – go to elsewhere, we do not want you here. I have reffered to Bruxelles, because this place is in CEE region seen as source of such ideas on regulation.”  OK, that message certainly didn’t come across at all but it does reflect the cultural environment that the organizers exist in.
7. There’s a surprising amount of support for the Agilia organizers.  You only need to look at @Agiliaconf’s Twitter feed to see this.  A lot of this support is coming from people in the CEE (whom the conference is targeted at) and in some cases even coming from…. you guessed it… women.  The point is that this gender diversity concern, and it is an important concern, is coming from outsiders who have a different cultural context than that of the target audience of the conference.  Once again, we’re seeing cultural differences get in the way of mutual understanding.
8. Acceptance of talk proposals from women was 100%.  One of the things that I asked the conference organizers was to provide some stats around their proposal process.  They received one talk proposal from a woman for this event and they accepted it.  Furthermore they reached out to another lady to give a talk at the conference.  Apparently she had been scheduled at a previous event but had to drop out due to personal reasons, so for this event they reached out to her again and invited her to speak at the Olomouc conference.

So, by choosing to have more respectful interactions with the conference organizers I easily discovered that they aren’t the evil, misogynistic bastards that some people want to portray them to be.  They are clearly guilty of having poor Twitter skills,  imperfect English, and a culture that is different from that of their critics.  I’m not sure that we should vilify them for that.  With a bit of investigation I discovered that they have a gender diverse conference committee, had not only accepted all of the proposals from women but had gone out of their way to invite another female speaker.  Could they have invited more women?  You bet.

We Need to Do Better

The organizer’s of the Agilia conference certainly made some serious mistakes.  But so did many of the people criticizing them.  Yes, the Agilia people reacted poorly, but that didn’t mean we needed to respond in kind by trying to publicly shame or in some cases bully anyone involved with the conference.

If I may be so bold, here are a few suggestions to consider in the future:

1. Let’s strive for respectful communication over indignant posing.
2. When we see questionable behaviour, let’s investigate first so that we can avoid jumping to conclusions.
3. Contact people privately to discuss and understand the situation before attacking them in public (particularly when you might not understand the full picture).
4. Just because others have jumped on a bandwagon, that doesn’t mean that you need to do so too.  Or, at least if you do, make sure you understand what’s actually going on first.
5. If you honestly believe that future Agilia conferences could do with a more diverse group of speakers, then take the opportunity to submit a proposal or point someone whom you think would be a great speaker to this link.
6. When someone is behaving in a way that offends you, for whatever reason, try to see it from their point of view first.  Yes, that means you’ll likely need to have a conversation with them rather than simply join the stone-throwing crowd.

In short, if you believe it’s appropriate to vilify someone, then at least have the integrity to make sure that they’re actually a villain before doing so.

Written by Kaushik Saha, Certified Disciplined Agile Coach (CDAC)

Many people focus on Scrum process framework when they talk about Agile because of several reasons:

1. Scrum is a lightweight method
2. Scrum is an improved process framework
3. Scrum is a very focused approach and event-driven
4. Scrum is easy to understand

But we must extend beyond Scrum wherein agile can be implemented in lean way and the Scrum could be one approach for the execution the construction phase.  Disciplined Agile Delivery (DAD) takes a “hybrid” approach and beginning-to-end approach that extend Scrum’s Construction lifecycle to explicitly address the full delivery lifecycle.  Disciplined Agile calls out three light-weight phases:

1. Inception –> Inception is a pre-Sprint phase in terms of Scrum which can talk about envisioning and planning where you  develop a common vision with stakeholders ; initial release planning; initialize the environment and infrastructure; identify initial risks; and adopt a goal-driven, non-prescriptive approach to development.
2. Construction –> During Construction the team incrementally and iteratively builds business value in the form of a potentially shippable, or better yet consumable, solution each iteration.
3. Transition –> During Transition you ensure that the solution is ready to be shipped and then ship it.

 Comparing Scrum and DAD:

1. Scrum delivery is focuses on “working software” but DAD goes further for a “complete end-to-end solution“.
2. Scrum is prescriptive, but DAD is pragmatic.
3. DAD is easily tailored.
4. Scrum focuses only on “Construction” phase; but DAD includes Inception. Construction, and Transition phases.
5. Scrum is targeted from single team to multiple teams. DAD goes further to be scalable at both the tactical and strategic levels.
6. Scrum is one approach process framework, but DAD is a Hybrid Framework that includes several lifecycles:  Agile (Scrum-based), Lean (Kanban-based), Continuous Delivery:Agile, Continuous Delivery: Lean, Exploratory (Lean Startup), and Program (team of teams).

The DAD Approach brings Agile and Lean Practices under one umbrella:

• DAD applies Lean development principles to enable scaling, wherein system can be optimized as whole by eliminating non-value added activities and build the quality inside with amplifying learning.
• DAD also adopts visualization of Kanban workflow to measure & manage flow of work by limiting work-in-progress.
• DAD’s Agile lifecycle applies the Scrum process framework in the Construction phase.
• DAD brings XP engineering practices in the Construction phase to reduce feedback cycle and develop quality code faster.