In this article, I will be referring more towards the lifecycle of testing involved into Agile. Please Note the article would be inferencing words from the agile world. Since I may not be defining them here, hence an explanation of those have been provided via source links.
When to Begin?
As part of an agile team, before kickstart of any testing activities, it’s important to understand the release plan and planned user stories across the sprints in a release.
A storyteller[a document enlisting the features or modules to be implemented in the upcoming releases] will be provided by the product owner
which would be compiled into multiple epics
and users stories
. All user stories are distributed among sprints based on their priority, business needs and team capacity by the product owner. Post awareness on the release plan and features, the first building block of testing activity starts with a test plan.
A test plan is created for each release and needs to be updated post every release. The main purpose of creating the test plan before the start of a release is to outline the scope, test approach, entry and exit criteria for the release. The exit criteria become the acceptance criteria for the release to be pushed to UAT and approved by the product owner.
For details on how to create a test plan, please refer my article Test Plan
The next phase of testing activity starts in sprint planning
. The team sits together to discuss the efforts involved across the planned user stories in a sprint and any dependencies involved in terms of development or testing. The team coordinate together to achieve the required velocity
of the sprint. Efforts across a user story are defined via story points, which are the efforts involved in both development and testing. Therefore during sprint planning estimation across user stories for the testing efforts are provided.
On how to perform test estimation please refer my article Test Estimation
[The article emphasizes test estimation to be done across modules, the same can be extended across user stories.]. Please note story points across user stories always follow Fibonacci series[1,2,3,5,8]
Post sprint planning, work allocation across the sprint is designed, wherein the testers involved in the project are assigned user stories in the sprint for test case creation and execution
Implementation and Execution
The next testing activity involved in agile is the creation of test cases across the assigned user stories. Post creation of user stories, peer review is performed. This activity provides a two-way value wherein the other QA gets awareness across the feature(user story) and helps to provide inputs for better test coverage. All test cases created in the test management tool must be linked across the stories. This helps to ensure all stories have the required test cases across them. For a better quality of deliverables, developers can view the test cases across the story and ensure on an overview that the code handles the scenarios enlisted, This helps to ensure a smaller cycle and lesser bugs across the story. Basically a quick delivery. Please note, as a good plan rule, all test cases should be created when the stories are under development. Once the stories are ready for testing, only execution across the story should be the left out task. This helps speed up the agile process. Creating test cases post the stories are ready for testing, just adds on the time for the closure of the story.
Post creation of test cases, once stories are ready for testing, the test cases created should be executed across it. Another good practice while execution is to ensure, the assigned tester who created the test cases should not be executing them.
The reason behind it is, the person would just execute the test cases and won’t explore the further area via Ad-hoc testing. Since as per him all test cases across the features have been written and he could not think more out of it. But a person who is executing the test cases may come across more pointers and may perform ad-hoc testing also. In an agile environment, ad-hoc testing plays an important role due to the shrinking testing cycles available. During execution, all testing types should be implemented be it functional, UI, usability, cross-browser etc. You can refer to my article on how to ace your execution skills
All bugs encountered across stories should be logged in the bug management tool and linked with user stories as well as test cases. A triage meeting is conducted to propose a plan for the bugs logged and their fix.
Post the bugs are fixed, they are retested across the stories and closed
Sprint closure activities
Once all the bugs linked to the stories and test cases are retested and closed, the user stories are also closed and are made ready for the product owner to be accepted. A sprint review
is performed demonstrating the deliverables of the sprint. If any story within a sprint does not gets closed due to outstanding bugs or development work pending, those get overflow to the next sprint. It is vital to understand a story is considered a deliverable only post testing activities are closed and not just with mere development work completion. A defect analysis report will be created post every sprint.
On details on how to create one, please refer my article Defect Analysis Report
. This report can also be created once post the release rather than across each sprint end. A sprint retrospective meeting
would be conducted to discuss what went good, bad and what should be done to ensure better productivity of the next sprint.
Post all sprints closure in a release, regression testing will be performed. A regression suite would be created from the test cases of the features planned in the release along with the impacted test cases of the previous releases. The regression suite would be approved by the product owner. Based on the approval, a regression estimate would be prepared before kick-starting regression that would outline the time required to complete off regression.
Daily status emails would be shared with the stakeholders highlighting the execution status and the number of bugs encountered. Depending upon the number of bugs and complexity of business involved, the regression may be divided into multiple cycles. Retesting across the bugs would also be performed in those cycles. Post regression completion a sign-off report would be created and shared with the stakeholders. If required a cumulative defect analysis report for all sprints will be also shared along with it.
Post a release closure, the build is deployed to the UAT environment and is made ready for few users to perform user acceptance testing. After deploying the release to UAT, a smoke is performed by the alpha testers and post smoke success, the release is given to the beta testers who test the build before signing off from there end to be shipped to production. Around a week time may be provided for beta testers to complete off testing. Once testing is completed, the build is been deployed to production and is made ready for end users to be tested after performing a round of smoke by the alpha testers. Any bugs encountered during UAT or production release would be logged and provided quick fixes if possible or a point release would be planned for the issues raised.
During this time other members of the testing team would be performing root cause analysis across the bugs raised in UAT or production.
Getting Ready for the next release
Post completion of the release, the team would prepare themselves for the next release, which would involve going through the storytellers encapsulating the features planned up for the release. Any business clarifications or issues would be raised and cleared during this phase.
The below image highlights the testing cycle and deliverables followed in the agile environment:
|The lifecycle of Testing in Agile
Closing the article with the below quotes, showcasing the agile practice:
“Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning.”– Albert Einstein
As agile organizations, teams and team members, we must constantly question what could be better in order to continually improve.