Importance of Configuration Management in Testing

You would have come across situations, where you have no clue or idea as the bug you see in the application now, was working earlier or the bug which was fixed earlier is now reopened. It becomes difficult to track from what point it was working and from which point it stopped working. Similarly goes to your test artefacts, where which test cases/scenarios of your regression suite were executed in which part of the release and which were not. Performing root cause analysis becomes difficult and almost an impossible task owing to such issues. So what is the key that resolves these issues? What is the factor which is important to ensure such issues are not encountered? Configuration Management

What is Configuration Management?

Configuration Management is the process of controlling the changes in the software through its complete lifecycle and tracking it to its closure. It encourages version controlling and establishment of the baselines. Configuration Management applies to all phase of SDLC. Development or testing both require, controlling the changes in software via version control. For the testing cycle, configuration management is implied to test cases, test plan, test execution, regression suite, bug logging etc. It basically covers every possible artefact and defect management cycle.

Let’s talk about how configuration management participates in software testing and its importance.

Before we kick-start, let’s understand how development before releasing the code to the testing environment, version control their code.
All major branches before been merged into the master branch in gitlab are tagged with the version, so that developer can backtrack as what code/features have been merged in the master and when. Post the features are ready and build is test ready, the development team provides a release note to the testing team, regarding the current build features.

Branches merging to master are version tagged

Release Notes: It is a document highlighting the features, modules and bug fixes release into a particular release deployed by the development team to the testing team. IEEE 829-Test item transmittal report also known as release notes is the document that contains all features to be tested by the testing team. Ideally and by standard, no testing should start without the release note, as that ensures that the listed features developed are released for testing. Also note, release notes are also delivered to end users, to display what has been released in the current version. Be it your android version/ios app updates or web application updates.

As far as testing is concerned in an agile environment, all user stories are tagged with the sprint name and release to which they are targetting to be delivered for. Based on these user stories all test cases are also tagged by the same version number. The question is why do we do so and what benefits does it reap in the longer run. So basically while creating a regression suite, cherry picking test cases from different features help ensure test cases across releases are selected and tested. While performing root cause analysis, it’s easy to locate the defect area and its origin from the release version. It also helps to identify the number of builds or release versions the test case was executed into.

As you can see in the current snapshot, the execution of the test case is version controlled via build number(release number).

The same goes out to the smoke document. The smoke document may be updated from time to time as per the business needs. It’s important to maintain the release version across every smoke document so that one is aware, what specifically was run in the smoke document and in which release. Release versions help in maintainability of the test artefacts. Any report like defect analysis report, test summary report or sign off report should always have baseline version and the approved version across it. A baseline version is a version the author first created the report with. Post that, based on the number of review cycles the report is updated with the approved version. For example, the baseline version for a sign-off report could be V0.1, post the review cycle, let’s say 2, the approved version would become V0.3. This helps to validate the number of review cycles the report has gone through and the version for which the report is prepared for.

Smoke Document Version Controlled

Defect Analysis Report version controlled

Document History inside reports should be version controlled

Version controlling and maintaining is also applicable to the bugs. It’s important to ensure all bugs are logged against the version they are reported and the version they are fixed into. Such parameters help in tracking as when the bug came into existing, when was it fixed and if required when was it reopened. While reopening issues it helps to validate as which part of the build, the bug belongs to.

Bugs logged with reported and fixed versions across them

Configuration Management plays an important role in testing, be it the test artefacts maintenance or in the analysis of issues/defects. Everything which is version controlled provides better visibility to the software. It helps in backtracking issues with lesser confusion and lesser efforts. Configuration Management also helps customers to have better views across features delivered in releases. It becomes easier to get acceptance across the release versions rather than just across the features. With adding complexity to the project and multiple environments, the only factor that could help in management and tracking is configuration management

Enclosing with an eye-opening quote:

“The problem is not that testing is the bottleneck. The problem is that you don’t know what’s in the bottle. That’s a problem that testing addresses.“ Michael Bolton

Author: Sadhvi Singh


Leave a Comment