Lesson Learnt - Requirement Changes in the middle of testing should be taken up as a Change Request

Requirement changes are nothing new to us.

I remember a recent situation.

- I was testing an interface and raised a defect.

- The developer on one application believed that the problem was on the other application side.

- Since this was an interface, the issue had to be resolved on either application.

- The defect fix was finally identified as a requirement change.

However, unfortunately this new requirement was not to be fixed/tested in the next release or a Change Request. It was decided that the current testing should continue and the new requirement be tested as soon as it was fixed. The QA raised the issue that the entire testing that has already finished will need to be repeated after the fix became available.

The project management still decided to continue with the fix in the same release.

Impact - 

1. QA did risk based re-testing after the fix was available. A lot of overtime hours were added to the budget.

2. Since the test scope was pretty much retested in a very short time, this is a risk to the production.

In other words, the project deferred the risk of defects to the production support.

Although in reality, we did not see any critical issues in production but the risk of errors stayed alive for a long time. The risk actually multiplied as the software rolled into 12 pilot stores from one test environment.

Hence, we learnt a lesson. The requirement changes that are identified in the middle of testing should be taken up separately as a separate release or a CR.

Lesson Learnt - In case of conflict with DEV, let BA and business confirm the defect and its severity

Testers are frequently in disagreements with developers over the validity of a defect. Developers defend their code and understanding of business knowledge. In some organizations, the skill level of developers is measured by the lesser number of defects found in their code. 

And this starts the disagreements like the following - 

1. This is not a defect. This was not a requirement to begin with.

2. This is not a defect. This is not what the requirement says.

3. This is not a defect. This is a design change. This cannot be accomplished in the given amount of time.

4. Why is the severity of this defect Major? This defect does not stop any processing.

5. This is existing functionality. It has always been like this.

It is quite human that we defend our hard work and what we think is right. And a developer has a technical or a coding angle from which he analyzes the quality of his work. In the real business world, he has a lot of potential to miss a requirement or the accuracy of a requirement. That is why they say - Developers cannot be good testers.

Hence, a lesson is learnt. If there is a disagreement between the tester and the developer, we should have a proper process in place.

- Consult the business analyst and get his opinion.

- If there is still a disagreement, consult business. The word of business should be considered final.

Lesson Learnt - QA should be involved early in the project (V Model of testing)

As the software projects become more complex, it is very important that QA gets involved in it as early as requirements gathering.

Benefits from the V-model -

Defect Prevention

If QA gets involved in requirements discussions, defect prevention happens naturally.

• As requirements are being built, QA provides its feedback and helps in correcting mistakes in requirement gathering.
• QA walk thrus the requirement documents along with business stakeholders and point out technical or business issues based on prior test experiences.
• We see QA provide its comments on the solution on many occasions. Sometimes, extending an existing solution proves cost effective than starting a brand new solution.

Informed QA is a better tester 

The QA becomes more informed in the requirements gathering and subsequent project phases. A more informed QA will definitely prove to be a better tester in the future.

Saves Time and Learning

In a waterfall project, QA start test planning after the requirements document has been signed off. The activity of knowledge transfer is a very lengthy process. If the resource on boarded is not the same as the one in prior release,  the ramp up time could be very high. However if the QA was involved in requirements discussions, there is a good possibility that there would be a small or no learning curve.

QA and BA in sync

When QA starts test planning and design, the critical and high priority requirements have already been identified. At this stage, the QA lead already has a good idea of what his test strategy is going to be.

Relevant Links on web

http://en.wikipedia.org/wiki/V-Model_%28software_development%29

http://www.testingexcellence.com/v-model-in-software-testing/