Regression Testing in Scrum
| When changes are made to software that has been previously validated, even the smallest revision has the potential to disrupt the existing code. Software developers implement Regression tests to ensure that modifications or additions to the code didn’t break any of the existing functionality. Another objective is to make sure that earlier variances are not re-introduced into the code inadvertently. Regression tests are used to show that automated tests that ran successfully the first time, will continue to run after code changes. This practice will ensure that the software has not regressed or been broken. This would mean that software components that were initially error-free have become defective due to unintentional code changes. With Agile projects such as Scrum, the repetitive activity of running a suite of Regression tests can be very complicated and lengthy. Without the utilization of automated testing tools, referred to as Regression Testing tools, agile testing would involve a high level of complexity. Figure 1. outlines the Regression testing event that occurs in Scrum during a Sprint cycle. After development and when a product increment is added, Integration Regression Testing takes place. The code is integrated with the existing product and verified to determine if the addition did not “break” anything. A Scrum software development best practice is to re-run the appropriate regression tests after every code change to ensure that previous tests continue to pass.
Figure 1. Regression Testing in Scrum Regression Testing Strategy It is believed, before Agile Methods because so popular, that Regression testing was easier. The complexity comes from Sprints that are often two weeks in duration and even shorter. Every time a software change occurs, the probability for new issues to be introduced into the software increases. Regression testing is the method used to uncover surprises in the code. An effective way to approach regression testing is by means of a “Regression Board”. Figure 2. shows a depiction of the board.
Figure 2. Regression Board When using a Regression Board, each member of the Scrum testing team needs to identify a change that they worked on during a Sprint. Once all changes have been identified, they are arranged in a top-down order based on which ones should be tested first. The order can be based on risk levels, customer importance, or any reason applicable to the project. Other team members reorder the items based on technical or value-driven information that is known to them. Different team members add their own distinct project perspective when they rearrange the items on the board. As the work is completed, pre-release testing commences. The Regression testing team refers to the Regression Board to make educational decisions about removing items from the list, adding additional staff to test low-risk items or possibly extending the release itself. As the testing team utilizes a distinct and organized list of regression testing items, it creates a major shift in thinking from “We don’t have enough time” to “We know what’s important and how much time we have”. Regression Automated Testing Tools When developers create new features, they can do their testing with automated Testing tools. This practice helps decrease risk when making code changes and results in the creation of a change detection system. Scrum projects have their own specific challenges for automation: o Project scope not clear o Multiple Iterations o Light documentation o Frequent and early automation requirements o Stakeholder involvement With traditional test automation, such as “test at the end” tools, also referred to as record and playback tools, a test team could not perform regression testing until after all software was developed. These types of tools don’t really support Scrum environments because there are better suited for Agile methods. Test automation at the beginning of the agile project is extremely difficult, however, as the system is completed, it then starts to become more relevant for automation. The Regression Test Suite - User Stories With Scrum, the user stories are selected for each Sprint and they used as the foundation for the test sets. As the Scrum team goes through each iteration, regression testing needs to be conducted to ensure that the existing functionality has not “broken” by the addition of new functionality in each Sprint. As the test team completes the Sprints, the magnitude of the regression testing increases. This is another instance of where good communication is needed among the Automation Testing and Scrum teams. The interaction between the client and the delivery team must remain collaborative. With heavy client involvement, comes more changes and more channels are needed for communications. The Test Automation methodology needs to ensure that the entire system, as it is developed, meets the business objectives for which it was intended and is fit for its purpose. Defects are automatically captured when an automated test script fails. Exploratory Testing With Agile Methods, exploratory testing demonstrates that the tester has value as an important component of the test process. In addition, values from the Agile Manifesto are shared: o Individual and interactions over processes and tools o Working software over comprehensive documentation o Customer collaboration over contract negotiation o Responding to change over following a plan With agile projects, automated tests that support code are mainly done by the development team and are executed from a programmer’s viewpoint. Exploratory tests are designed to find defects that are beyond the scope of automated programmer tests. The exploratory tests focus on possible gaps where automation may be weak. These tests are not structured and are considered “freestyle”. In comparison, Automated tests are used more for regression testing and Exploratory tests are used for new features. Test-Driven-Development (TDD) TDD is a process that begins with creating tests for small pieces of functionality. Tests are developed before the code and it contains the specification and the validation of what the program needs to accomplish. TDD runs automated tests prior to the development of the software. TDD is often referred to as Test First-Development (TFD). Finally, TDD is the practice of modifying the code so that a test that was previously designed will pass. The TDD process is as follows:
Figure 3. outlines the TDD process.
Figure 3. Test Driven Development (TDD)
Keywords: regression, testing, Scrum References: Guru99. (2017). Test Driven Development: Learn with Example. Retrieved from https://www.guru99.com/test-driven-development.html#1 QASymphony. (2016). Developing a Regression Test Strategy. Retrieved from https://www.qasymphony.com/blog/developing-regression-testing-strategy/ QAT Global USA. (2017). Agile Meets Scrum in QAT Global’s Enterprise Development Framework. Retrieved from http://www.qat.com/agile-scrum-methodology/ SmartBear Software. (2017). What is Regression Testing? Retrieved from https://smartbear.com/learn/automated-testing/what-is-regression-testing/ Software Testing Help. (2017). Automated Regression Testing Challenges in Agile Environment. Retrieved from http://www.softwaretestinghelp.com/automated-regression-testing-challenges-in-agile-testing-environment/ Testing Excellence. (2017). Why Exploratory Testing is Important in Agile Projects. Retrieved from https://www.testingexcellence.com/exploratory-testing-important-agile-projects/ |
Task Estimation with Scrum
| The Sprint Planning Meeting The Sprint Planning meeting is the Scrum ceremony where the team estimates all tasks associated with each user story in the Sprint backlog. The estimation effort focuses on all resources required to complete the tasks for each Sprint. A shared task list is used to estimate the duration and effort for each user story in the backlog. This practice results in a shared team perspective, resulting in a task estimation process that is consistent and reliable throughout all Sprints. Estimation Techniques The most commonly used Scrum estimation technique is the practice of using story points to represent the value of a user story or task. This method is used for estimation based on the relativity of a backlog item to similar items. To be clear, many Scrum teams use story points for task estimation, but it is much more common to use hours as the unit of measure for tasks. A story point can be set for any value that a single Scrum team decides upon. Some teams assign the value of 1 week to a story point and other teams set the story point value to 1 hour or 1 day. There is no established value for a story point, however the value should be consistent throughout a Scrum project, or least within the same team. For example, if it has been decided that a story point has a value of 1 day, then 3 story points would equal 3 days. Figure 1 shows the relationship between epics (large user stories that need to be further broken down), user stories (requirements) and tasks (actual work to be done to develop the user story). Story points do not measure how long it will take to complete a user story or task. Story points are typically based on the Fibonacci sequence which is a number sequence where every number is the sum of the previous two. The sequence goes like this: 0, 1, 2, 3, 5, 8, 13, 21, etc. This technique is called Scrum Planning Poker where the Scrum team works to create accurate estimates. Planning Poker does not represent “hourly” estimates because a story point is an abstract measurement of size. Story points help understand relative levels of effort (LOE) for a user story or task in comparison to other stories and tasks.
Figure 1: Backlog items Decomposition of Tasks The task list contains all assignments that are needed to complete the user stories in the Sprint Backlog. High-level tasks are decomposed into smaller tasks with greater detail. The level of decomposition should be sufficient to create product deliverables from the items on the task list. Task Dependencies Dependencies between the user stories and tasks should be evaluated, including those that are related to technical and staff availability. There are several types of dependencies that need to be discussed:
Estimate Tasks in Ideal Time To recap, estimation criteria is expressed in two ways, story points (a measure of size, risk and complexity) and ideal time (a measure of time with no interruptions). The concept of ideal time describes the number of hours or days that are required to complete the work for the project’s deliverables. It is important to understand that ideal time does NOT include any time for work that is outside of the project’s scope. Ideal time represents the interval needed to do work without any interruptions (phone calls, lunch, meetings, etc.). Ideal hours are used to provide “time-based” estimates for development tasks. It is common that most estimates are excessively inaccurate where ideal estimates typically have realistic durations. Let’s look at how ideal time is calculated. We will look at ideal time in terms of days and hours. 9 developers on a Scrum Team
To summarize, ideal time represents how long a task takes if:
Elapsed time represents the total amount of time used to finish project work. Ideal time and elapsed time are not equivalent. If estimates were based on elapsed time, then ALL interruptions would need to be included. There are several advantages and disadvantages in using ideal time:
Key Words: Tasks, Estimation, Scrum References: Canty, Denise. (2015) Agile for Project Managers (Best Practices and Advances in Program Management). Boca Raton, FL: CRC Press Cohn, Mike. (n.d.) Chapter 8: Choosing between Story Points & Ideal Days. Retrieved from http://athena.ecs.csus.edu/~buckley/CSc231_files/Cohn_Ch8.pdf SCRUMstudy. (2016) A Guide to the Scrum Body of Knowledge (SBOKTM Guide.), 3rd Edition |
Scrum Guidance Body Recommendations
| The Scrum Guidance Body (SGB) typically consists of a group of professionals that define the set of standards that pertain to quality, government regulations, and other important organizational considerations. The standards are developed to provide guidance for the Product Owner, Scrum Mater and the Scrum Team. The SGB also is responsible for capturing Scrum Best Practices for use by all Scrum projects within an organization. Decision-making at the project level is not supported by the SGB; rather it provides guidance and/or the consulting foundation for all organizational project related levels (i.e. portfolio, program, and project). The Scrum Team is free to utilize the SGB as they need advice. Recommendations from the Scrum Guidance Body should be used on Scrum projects to ensure of the proper alignment of the project vision and process compliance to the standard and guidelines that have been established by the Body. Although the SGB is an important role, it must be understood that the Body is optional. Scalability Recommendations Scalability refers to the ability to adapt to any type of expansion. For Scrum, it means that a single Scrum Team can be scaled for larger projects by means of multiple teams. The Guide to the Scrum Body of Knowledge (SBOK Guide) has provided specific steps that support Scrum’s ability to be scaled and applied on large projects. Scalability in Scrum can be applied at three levels: Projects, Programs and Portfolios. The process begins with the Scrum of Scrum (SoS) meetings that support harmonization between multiple Scrum Teams. A representative from each of team provide feedback regarding their team’s progress, issues confronted and synchronization activities. The frequency of the SoS meeting is based on recommendations from the SGB, complexity level, project size and dependencies between the teams. Other recommendations include co-location and face-to-face communication among the Scrum team. As many of us may have experienced, co-location is sometimes very difficult to achieve. Companies often use distributed teams that work in different time zones and a variety of geographies. Regarding scaling for large projects, geographically dispersed team use chats, social media, video conferencing and other virtual communication practices. Large Project Considerations With projects that create large components, it is important to understand the role of multiple Product Owners and the way that multiple Scrum Teams work together. We will now discuss the inputs that are necessary for creating large project components in Scrum. Basically, the role of the Product Owner stays for same for small and large projects. The difference is that for large projects, the PO will not make daily decision a priority. Instead, the PO provides input and recommendations to the Chief Product Owner. Stakeholder interactions are distributed between all Product Owners and each one continues to work with their designated team. Role and responsibilities are captured in the Product Owners Collaboration Plan. Large project planning often results in recommendations to revise or improve the Scrum Guidance Body recommendations. If the body accepts the proposed modifications or additions, they will be added as updates to the SGB documentation. Figure 1 outlines the workflow for the creation of the large project components.
Figure 1. Create Large Project Components – Data Flow Diagram Chief Product Owner The Chief Product Owner is responsible for making daily business decisions on large projects. This role is responsible for the coordination of work for multiple Product Owners. With feedback from the POs, the Chief PO is responsible for the preparation and maintenance of the Prioritized Product Backlog, which is used as the source of work for the Product Owners and their Scrum Teams. Finally, the Chief PO takes care of the final deliverable for the projects. Lastly, each PO for the Scrum Teams is responsible for only the component and features that are developed by their assigned Scrum Teams. Project Vision The project vision clarifies the business need for the project. This statement should not be specific and needs to have room to be very adaptable. The reasons behind this flexibility are because it is very probable that the project could be based on suppositions that could change as the project progresses. The project vision must be able to accommodate changes and it should focus on the problem and NOT on the solution. Chief Scrum Master The Chief Scrum Master has the responsibility for communicating information and managing dependencies between the Scrum Teams on large projects. Collaboration is required among the Scrum Team via the Scrum of Scrums (SoS) meetings. One of the main responsibilities of the Chief Scrum Master is to remove impediments and fostering a productive environment for the Scrum Teams. This role also collaborates with the Chief Product Owner, Scrum Masters, and Product Owners to establish a list of components and resources needed across all Scrum Teams. As expected, the Chief Scrum Master is expected to facilitate all ceremonies that exceed the responsibilities of a single Team Scrum Master. There is also consistent interfacing with the Program Scrum Master to make sure that there is the proper association of the large project with the goals and intentions of the related program. Large Project Environments For large projects, it is important to determine the number and types of environments needed for a large number of Scrum Teams to accomplish their work during their Sprints. These categories of environments include but are not limited to testing, development, work locations, resources or applicable procedural borders required for the Scrum Teams. Definition of Done The SGB generally defines and documents the Definition of Done. The “Done” criteria represent a set of rules that will be applied to all user stories in a specific Sprint, including but not limited to the following:
References: SCRUMstudy. (2016). A Guide to the Scrum Body of Knowledge (SBOKTM Guide.), 3rd Edition SCRUMstudy. (2017). Scalability of Scrum. Retrieved from https://www.scrumstudy.com/blog/scalability-of-scrum/ SCRUMstudy. (2017). Inputs Required for Creating Large Projects in Scrum. Retrieved from https://www.scrumstudy.com/blog/inputs-required-for-creating-large-project-components-in-scrum/ SCRUMstudy. (2017). What are Done Criteria? Retrieved from https://www.scrumstudy.com/blog/what-are-done-criteria/ |
Metrics and Measuring Techniques in the Retrospective Meeting
| A retrospective is a time-boxed ceremony where the Core Scrum Team (optional for the Product Owner) convenes to discuss the iteration that was most recently completed. This practice is very similar to the lessons learned meeting that takes place in waterfall projects. Pertinent information is collected during these meetings and is documented for utilization in future Sprints. Scrum Team members discuss their existing best practices, potential improvements, issues and blockages. The Scrum Master ensures that the high priority recommendations are implemented not later than the next Sprint. The retrospective is an Inspect and Adapt event that is facilitated by the Scrum Master. Discussions, based on what went right and what went wrong, are recorded for imminent accomplishment. Participation by all team members is expected. The main goals of the retrospective are to provide responses for three distinct assessments:
Metrics and Measuring Techniques There are a variety of metrics that a Scrum team can utilize to measure their performance on a Sprint by Sprint basis. These metrics have been identified in Table 1. below.
Table 1. Retrospective Metrics Velocity Velocity is a number that stands for the average number of user stories that have been completed during the Sprints. When a Scrum team has determined the average number of story points that they can complete, they can then calculate the estimated time frame that it will take to finish the project. The team will use the number of user stories that need to be completed and then divide this number among the remaining Sprints. For example, if a team has a total of 150 story points remaining, then the projection for completion is then 9. This assumes that the average velocity has been averaged at 17 story points per Sprint.
Completed Success Rate This metric is represented as a percentage of the story points completed based on what the team projected that they would complete. For example, if the team made a commitment to complete 50 story points and they only completed 49, the completed success rate would be 49/50 = 98%.
Estimation Accuracy This metric is represented as a percentage of the actual time spent on tasks and user stories and the time that the team estimated would be needed. For example, if the team estimated their total work as 50 hours and it took 45 hours to complete, then 45/50 = 90% estimation accuracy.
Feedback Ratings This metric is the feedback rating from the stakeholders on the project using subjective and/or objective ratings that measures the Scrum Team’s performance. For example, stakeholder may provide feedback as “Very Good, Excellent, or Outstanding”. This would be an subjective measurement of feedback. On the other hand, if a stakeholder responds on a scale of 1 to 5, where:
The above measurement represents an objective feedback rating.
Team Morale Ratings The Scrum Team members conduct self-assessments regarding their morale in relationship to the project. For example, team members provide information such as:
Peer Feedback This metric is used to provide feedback for Scrum team members where each team member choses a peer, conducts observation during an agreed upon time frame and then share the information to the selected peer. With Scrum, other members of the Core Scrum Team can participate in peer feedback (Scrum Master, Product Owner). Some firms use the 360-degree feedback model, a many-to-many feedback format. This model focuses on team member performance evaluation and a questionnaire is typically used. This model has little to no team collaboration.
Release Progress This metric is based on the amount of business value that is provided in each release based on story points. A Release Burnup Chart is used to identify the work completed for the release. A business uses this metric to determine how much work has been delivered. Burndown Charts can also be used for this metric. See Figure 1. below.
Figure 1. Release Burnup Chart (Agile Velocity Blog, 2014) In conclusion, companies have their organizational specific measurement techniques and metrics. The metrics presented here are just examples used during the Retrospective meetings and the choice of which ones to use are based on the project’s needs. Keywords: retrospective, metrics, measurements
References: Agile Alliance. (2015). Glossary. Velocity. Retrieved from https://www.agilealliance.org/glossary/velocity/ Agile Alliance. (2015). Peer Feedback. Retrieved from https://www.scrumalliance.org/community/articles/2015/january/new-innovation-to-scrum-ceremonies-peer-feedback Agile Velocity Blog. (2014). Improve Your Visibility into Release Progress. Retrieved from http://www.agilevelocity.com/blog/release-planning/ SCRUMstudy. (2016). A Guide to the Scrum Body of Knowledge (SBOKTM Guide.), 3rd Edition |
Impediment Logs in Scrum
| Impediments are barricades, hurdles or obstacles. In terms of Scrum, they are “blockers” that prevent the Scrum Team from completing work, which in return impacts velocity. Anything that prohibits the team from doing work is considered an impediment. Examples of Scrum impediments are listed in Table 1.
Table 1. Scrum Impediments The Scrum Master is responsible for tracking, monitoring and ensuring that impediments are removed. All Scrum Team members are responsible for continually identifying impediments for discussion during the Daily Standup Meeting. If for some reason an impediment does not disappear in a timely manner, this would indicate that the root causes have not been identified. The Sprint Retrospective is another place for impediments that reoccur. It is important to understand that the Scrum Master is not solely responsible for the removal of impediments. The team should work together to remove impediments that can be easily resolved and provide assistance with any additional support that may be required. A few things to note:
There are two main types of impediments, organizational and team related and they need different types of handling.
The expectation is that the team can learn to remove its own impediments without the Scrum Master’s intervention. This also means that impediments in the log should not be delegated to the team because many of them may be very difficult to resolve. On the other hand, the Scrum Master is not expected to resolve all impediments alone either. The entire Scrum Team needs to work together to determine which impediments it can resolve and what support may be needed. Over time, the team should become capable of removing more and more impediments on its own. Impediment Logs There should only be a single Impediment Log for a Scrum Master to manage. Table 2. outlines the process that is typically used to create, monitor and maintain the Impediments Log.
Table 2. Impediments Process in Scrum Table 3. identifies a description of each of the field on the Log. Figure 1. is an illustration of an Impediments Log. Let’s examine the data input fields to gain the proper understanding of their usage.
Table 3. Impediment Log Data Input Fields
Figure 1. Impediments Log Tips for Removing Impediments Following are several tips for the removal of impediments:
Keywords: Impediments, Logs, Scrum References: CoreWorks. (2014). The Impediments Backlog. Retrieved from http://www.coreworks.co/scrum-impediments-backlog Getting Agile. (2011). Organizational Impediment Management: Early Risk Detection for Agile. Retrieved from http://www.gettingagile.com/2011/01/24/organizational-impediment-management-early-risk-detection-for-agile/ LeanAgileTraining. (2017). What are Impediments? Retrieved from https://www.leanagiletraining.com/impediments/what-are-impediments/ Openbravo wiki. (2009). Scrum/Impediment. Retrieved from http://wiki.openbravo.com/wiki/Scrum/Impediment Overeem, Barry. (2016). The Scrum Master as an Impediment Remover. Retrieved from http://www.barryovereem.com/the-scrum-master-as-an-impediment-remover/ Scrum Alliance. (2011). Five Tips for Impediment Resolution With Scrum. Retrieved from https://www.scrumalliance.org/community/articles/2011/september/five-tips-for-impediment-resolution-with-scrum scruminc. (2017). Impediments. Retrieved from https://www.scruminc.com/impediments/ | ||||||||||||||||||||||||||||||||||||||











