I am wondering what would be the best way to organize the work of a team of Data Scientists where each 2 people work on a different project? I was wondering if Scrum will still be a good idea - can we have one Product backlog and one Scrum Dashboard for multiple projects? Can we still have the Scrum events - one Sprint planning meeting, One Daily Standup - without having to organize multiple meetings for each 2 people? This is not exactly scrum of scrums since we don't have at least 5 people in a team. I haven't worked in such setup before and I am looking for people who have to advise on best practices. Maybe Kanban will also be applicable - or a combination of Scrum + KANBAN. Thank you in advance and the more detailed your answer - the better! Saving Changes...
I've found when you have a service team which has practitioners helping multiple project teams, a Kanban-based approach might work better than a timebox-based one like Scrum to visualize and manage the work of the different team members. This works well when the involvement of those practitioners is for short periods of time on each project.
However if the project assignments are long, each project manager will likely have the same needs and they would likely want the DS's assigned to their projects participating in their events (e.g. Daily Scrums) and their work would be tracked in the work management approach of each individual project.
In such cases, the people manager for the DS's would use an alternate workforce supply/demand tracking approach.
I agree with Kiron that a Kanban type approach is a more natural fit for what you describe. It was my immediate thought reading data science and teams of 2.
Those types of situations are often ill suited for either trying to force every project into either fixed time increments or maturity levels. The variance in the complexity between efforts is too large and those constraints create more work to fit square pegs in round holes. Kanban is often used in continuous improvement environments for that reason. It's easy to track different projects, some in their infancy, others nearing completion, some simple and some highly complex on the same visibility board.
That doesn't mean you have to do Kanban "by the book" as it were. Feel free to customize your approach to what works like whether or not to have daily standups. Communication needs between many small teams depends on frequency of change, and required interaction across teams. If everyone has to talk their line every morning and most have zero influence on other teams, you're most likely wasting other people's time. If efforts have lots of impact on other teams, it is often better to overshare. If you need info to explain your teams' progress to your bosses, you can still talk to them individually rather than death by status meeting.
Also beware of the the common pitfall of following the same approach forever. If the business environment changes and the process becomes the obstacle, change the process. Saving Changes...
By working with a team of data scientists where each pair of people work on a separate project, it is indeed possible to tailor Scrum to meet requirements. I would build the work according to such a plan for the organization of work:
1. Product backlog. Create a single product backlog covering all projects. It should include user stories or tasks for each project and be prioritized based on the team's overall goals.
2. Scrum dashboard. Create a shared Scrum dashboard or project management tool where the team can visualize and track the progress of each project. This will help everyone to have a clear idea of the current state of the projects.
3. Scrum Events: Conduct Scrum events at the team level, not the project level, to maintain a coordinated effort. Here's how you can tailor each event:
A. Sprint planning. During the sprint planning meeting, collectively decide which backlog items each pair of data scientists will work on in the upcoming sprint. Make sure each project gets the attention it needs and balance the workload between pairs.
b. Daily stand-up: Host a daily stand-up for the whole team, where each couple talks about their progress, obstacles, and plans for the day. This ensures cross-pollination of ideas and potential learning opportunities.
c. Review and retrospective of the sprint. Organize a sprint review and retrospective at the end of the sprint for the entire team. Each couple can present their project progress, achievements and challenges. This encourages collaboration and allows the team to learn from each other's experiences.
4. Kanban. Consider using Kanban in addition to Scrum. Kanban can provide a visual representation of the workflow for each project, allowing team members to track status and identify potential bottlenecks. This can be useful, especially when projects have different priorities or when tasks require different levels of effort.
The combination of Scrum and Kanban allows you to benefit from the Scrum framework while leveraging the flexibility of Kanban to effectively manage multiple projects.
To summarize, Scrum can be implemented in a team of data scientists working on various projects. Having a single backlog, a shared dashboard and team-level Scrum events, and implementing Kanban can help maintain coordination, transparency, and efficient workflow across all projects.
...
1 reply by anonymous
Sep 07, 2023 5:59 AM
anonymous
...
Hello Maxim and many thanks for the great response. In this scenario above, could you please tell me what would be the difference between the Scrum and the KANBAN dashboard so it would be beneficial for the team to have both? Both will help the team visualize the work for each project. I am not very experienced w/ Kanban and I want to make sure I understand :)
Saving Changes...
Anonymous
Sep 06, 2023 11:53 PM
Replying to Maxim Shevelev
...
By working with a team of data scientists where each pair of people work on a separate project, it is indeed possible to tailor Scrum to meet requirements. I would build the work according to such a plan for the organization of work:
1. Product backlog. Create a single product backlog covering all projects. It should include user stories or tasks for each project and be prioritized based on the team's overall goals.
2. Scrum dashboard. Create a shared Scrum dashboard or project management tool where the team can visualize and track the progress of each project. This will help everyone to have a clear idea of the current state of the projects.
3. Scrum Events: Conduct Scrum events at the team level, not the project level, to maintain a coordinated effort. Here's how you can tailor each event:
A. Sprint planning. During the sprint planning meeting, collectively decide which backlog items each pair of data scientists will work on in the upcoming sprint. Make sure each project gets the attention it needs and balance the workload between pairs.
b. Daily stand-up: Host a daily stand-up for the whole team, where each couple talks about their progress, obstacles, and plans for the day. This ensures cross-pollination of ideas and potential learning opportunities.
c. Review and retrospective of the sprint. Organize a sprint review and retrospective at the end of the sprint for the entire team. Each couple can present their project progress, achievements and challenges. This encourages collaboration and allows the team to learn from each other's experiences.
4. Kanban. Consider using Kanban in addition to Scrum. Kanban can provide a visual representation of the workflow for each project, allowing team members to track status and identify potential bottlenecks. This can be useful, especially when projects have different priorities or when tasks require different levels of effort.
The combination of Scrum and Kanban allows you to benefit from the Scrum framework while leveraging the flexibility of Kanban to effectively manage multiple projects.
To summarize, Scrum can be implemented in a team of data scientists working on various projects. Having a single backlog, a shared dashboard and team-level Scrum events, and implementing Kanban can help maintain coordination, transparency, and efficient workflow across all projects.
Hello Maxim and many thanks for the great response. In this scenario above, could you please tell me what would be the difference between the Scrum and the KANBAN dashboard so it would be beneficial for the team to have both? Both will help the team visualize the work for each project. I am not very experienced w/ Kanban and I want to make sure I understand :) Saving Changes...
When folks are working on different projects, the use of Scrum is limiting. What is your Sprint Goal or Goals? Who dictates priority - not your team's manager, that's for sure. What's the benefit of a daily Scrum when team members are working on different projects?
I actually thought a lot about this over the weekend while listening to webinars and doing home projects...
I would be cautious about how you try to combine the two including using both boards. It sounds like a recipe for a headache. The two are focused on different approaches, carefully managed time boxed increments, vs. visualization of flow, and Scrum has a lot more process rules. There are many excellent articles on the web that describe them side by side in detail.
A situation I often find with data science type projects is IT issues out of our team's control that put some projects temporarily on hold, like addressing security protocols which make it very difficult to access the data or systems updates that will pace some schedule items. In Kanban, you can grab other items out of the backlog and work them earlier while waiting or go help out another team which you don't in Scrum. Then velocity data is also becomes a bit meaningless when you can work some projects start to finish and not others. The clock doesn't stop ticking.
You can very easily integrate sprints into Kanban where it fits without doing Scrum, and you can always add other useful metrics to your dashboard. Be careful about trying to adopt too much of the process elements though when they were designed around a different context. Saving Changes...
AIP-DM (Agile Iteration Process for Data Mining), designed by Siddhesh Dongare, stands out as one of the most practical and execution-focused agile frameworks for Data Science and AI programs. What makes it different is that it does not treat Data Science purely as experimentation or traditional software delivery. Instead, it creates a balanced operating model across Mindset, People, Process, and Outcomes, which are often the missing links in enterprise AI execution.
The integration of agile principles with structured data mining practices helps teams bridge the gap between research, engineering, governance, and business value delivery. The S.T.A.R. Q.U.E.S.T. principles further strengthen the framework by bringing focus toward quality, ethics, scalability, sustainability, and measurable outcomes, which are critical for modern AI systems.
A key strength of AIP-DM is its real-world applicability. Unlike many theoretical frameworks, it has been successfully implemented within Mastercard Open Finance programs, demonstrating its ability to support large-scale, enterprise-grade AI and data initiatives across global markets.
As organizations increasingly struggle with aligning Data Science experimentation with production delivery, governance, and measurable business impact, frameworks like AIP-DM provide a much-needed structured yet agile approach for sustainable AI transformation.
