When changes are made to software that has been previously validated, even the smallest revision has the potential to disrupt the existing code. Software developers implement Regression tests to ensure that modifications or additions to the code didn’t break any of the existing functionality. Another objective is to make sure that earlier variances are not re-introduced into the code inadvertently. Regression tests are used to show that automated tests that ran successfully the first time, will continue to run after code changes. This practice will ensure that the software has not regressed or been broken. This would mean that software components that were initially error-free have become defective due to unintentional code changes.
With Agile projects such as Scrum, the repetitive activity of running a suite of Regression tests can be very complicated and lengthy. Without the utilization of automated testing tools, referred to as Regression Testing tools, agile testing would involve a high level of complexity. Figure 1. outlines the Regression testing event that occurs in Scrum during a Sprint cycle. After development and when a product increment is added, Integration Regression Testing takes place. The code is integrated with the existing product and verified to determine if the addition did not “break” anything. A Scrum software development best practice is to re-run the appropriate regression tests after every code change to ensure that previous tests continue to pass.
Figure 1. Regression Testing in Scrum
Regression Testing Strategy
It is believed, before Agile Methods because so popular, that Regression testing was easier. The complexity comes from Sprints that are often two weeks in duration and even shorter. Every time a software change occurs, the probability for new issues to be introduced into the software increases. Regression testing is the method used to uncover surprises in the code. An effective way to approach regression testing is by means of a “Regression Board”. Figure 2. shows a depiction of the board.
Figure 2. Regression Board
When using a Regression Board, each member of the Scrum testing team needs to identify a change that they worked on during a Sprint. Once all changes have been identified, they are arranged in a top-down order based on which ones should be tested first. The order can be based on risk levels, customer importance, or any reason applicable to the project. Other team members reorder the items based on technical or value-driven information that is known to them. Different team members add their own distinct project perspective when they rearrange the items on the board.
As the work is completed, pre-release testing commences. The Regression testing team refers to the Regression Board to make educational decisions about removing items from the list, adding additional staff to test low-risk items or possibly extending the release itself. As the testing team utilizes a distinct and organized list of regression testing items, it creates a major shift in thinking from “We don’t have enough time” to “We know what’s important and how much time we have”.
Regression Automated Testing Tools
When developers create new features, they can do their testing with automated Testing tools. This practice helps decrease risk when making code changes and results in the creation of a change detection system. Scrum projects have their own specific challenges for automation:
o Project scope not clear
o Multiple Iterations
o Light documentation
o Frequent and early automation requirements
o Stakeholder involvement
With traditional test automation, such as “test at the end” tools, also referred to as record and playback tools, a test team could not perform regression testing until after all software was developed. These types of tools don’t really support Scrum environments because there are better suited for Agile methods. Test automation at the beginning of the agile project is extremely difficult, however, as the system is completed, it then starts to become more relevant for automation.
The Regression Test Suite - User Stories
With Scrum, the user stories are selected for each Sprint and they used as the foundation for the test sets. As the Scrum team goes through each iteration, regression testing needs to be conducted to ensure that the existing functionality has not “broken” by the addition of new functionality in each Sprint. As the test team completes the Sprints, the magnitude of the regression testing increases. This is another instance of where good communication is needed among the Automation Testing and Scrum teams. The interaction between the client and the delivery team must remain collaborative. With heavy client involvement, comes more changes and more channels are needed for communications. The Test Automation methodology needs to ensure that the entire system, as it is developed, meets the business objectives for which it was intended and is fit for its purpose. Defects are automatically captured when an automated test script fails.
Exploratory Testing
With Agile Methods, exploratory testing demonstrates that the tester has value as an important component of the test process. In addition, values from the Agile Manifesto are shared:
o Individual and interactions over processes and tools
o Working software over comprehensive documentation
o Customer collaboration over contract negotiation
o Responding to change over following a plan
With agile projects, automated tests that support code are mainly done by the development team and are executed from a programmer’s viewpoint. Exploratory tests are designed to find defects that are beyond the scope of automated programmer tests. The exploratory tests focus on possible gaps where automation may be weak. These tests are not structured and are considered “freestyle”. In comparison, Automated tests are used more for regression testing and Exploratory tests are used for new features.
Test-Driven-Development (TDD)
TDD is a process that begins with creating tests for small pieces of functionality. Tests are developed before the code and it contains the specification and the validation of what the program needs to accomplish. TDD runs automated tests prior to the development of the software. TDD is often referred to as Test First-Development (TFD). Finally, TDD is the practice of modifying the code so that a test that was previously designed will pass. The TDD process is as follows:
- Add a test.
- Run all tests and see if any new test fails.
- Write some code.
- Run tests and Refactor code.
- Repeat
Figure 3. outlines the TDD process.
Figure 3. Test Driven Development (TDD)
Keywords: regression, testing, Scrum
References:
Guru99. (2017). Test Driven Development: Learn with Example. Retrieved from https://www.guru99.com/test-driven-development.html#1
QASymphony. (2016). Developing a Regression Test Strategy. Retrieved from https://www.qasymphony.com/blog/developing-regression-testing-strategy/
QAT Global USA. (2017). Agile Meets Scrum in QAT Global’s Enterprise Development Framework. Retrieved from http://www.qat.com/agile-scrum-methodology/
SmartBear Software. (2017). What is Regression Testing? Retrieved from https://smartbear.com/learn/automated-testing/what-is-regression-testing/
Software Testing Help. (2017). Automated Regression Testing Challenges in Agile Environment. Retrieved from http://www.softwaretestinghelp.com/automated-regression-testing-challenges-in-agile-testing-environment/
Testing Excellence. (2017). Why Exploratory Testing is Important in Agile Projects. Retrieved from https://www.testingexcellence.com/exploratory-testing-important-agile-projects/
Community Champion
