Requirements Tracebality Matrix

A matrix is a table that maps one variable to another.

A requirements traceability matrix or RTM maps requirements to test cases.

Example - Requirements Traceability Matrix (RTM)

Background

For any project, the business analyst builds requirements based on business needs. The BA prioritizes the requirements as critical (Level 1), major (Level 2) and good to have (Level 3).

Based on these requirements, the QA does his test design. For each requirement, the QA plans and scripts his test cases. At the end of this design, the QA builds a RTM and verifies that each requirement built by the BA will be tested by at least one test case.

That is the purpose of Requirements Traceability Matrix. The table what is RTM, clearly lays out what is called Test Coverage.

For different types of testing, there could be different requirements.

• System testing has module level technical requirements based on the solution for the business requirements.
• System Integrated Testing has business scenario requirements.
• User Acceptance Testing has user prioritized business scenario requirements.
• Performance testing has non functional requirements (NFR).

Sign Off decision based on RTM and test coverage

However in each case, a sign off authority would sign off the test design only by checking the test coverage based on the Requirements Traceability Matrix.

Go/No Go Decision

Another use of RTM is to take a Go/No Go decision. 

It is possible that in the planned amount of time, QA could not execute all scheduled tests. In this case, the QA lead and the BA can check the RTM and see which requirements were not tested 100%. Now based on the priority of the partially tested requirements, the leads can easily decide if the software is fit to move to production.

Lesson Learnt - Requirement Changes in the middle of testing should be taken up as a Change Request

Requirement changes are nothing new to us.

I remember a recent situation.

- I was testing an interface and raised a defect.

- The developer on one application believed that the problem was on the other application side.

- Since this was an interface, the issue had to be resolved on either application.

- The defect fix was finally identified as a requirement change.

However, unfortunately this new requirement was not to be fixed/tested in the next release or a Change Request. It was decided that the current testing should continue and the new requirement be tested as soon as it was fixed. The QA raised the issue that the entire testing that has already finished will need to be repeated after the fix became available.

The project management still decided to continue with the fix in the same release.

Impact - 

1. QA did risk based re-testing after the fix was available. A lot of overtime hours were added to the budget.

2. Since the test scope was pretty much retested in a very short time, this is a risk to the production.

In other words, the project deferred the risk of defects to the production support.

Although in reality, we did not see any critical issues in production but the risk of errors stayed alive for a long time. The risk actually multiplied as the software rolled into 12 pilot stores from one test environment.

Hence, we learnt a lesson. The requirement changes that are identified in the middle of testing should be taken up separately as a separate release or a CR.

Lesson Learnt - In case of conflict with DEV, let BA and business confirm the defect and its severity

Testers are frequently in disagreements with developers over the validity of a defect. Developers defend their code and understanding of business knowledge. In some organizations, the skill level of developers is measured by the lesser number of defects found in their code. 

And this starts the disagreements like the following - 

1. This is not a defect. This was not a requirement to begin with.

2. This is not a defect. This is not what the requirement says.

3. This is not a defect. This is a design change. This cannot be accomplished in the given amount of time.

4. Why is the severity of this defect Major? This defect does not stop any processing.

5. This is existing functionality. It has always been like this.

It is quite human that we defend our hard work and what we think is right. And a developer has a technical or a coding angle from which he analyzes the quality of his work. In the real business world, he has a lot of potential to miss a requirement or the accuracy of a requirement. That is why they say - Developers cannot be good testers.

Hence, a lesson is learnt. If there is a disagreement between the tester and the developer, we should have a proper process in place.

- Consult the business analyst and get his opinion.

- If there is still a disagreement, consult business. The word of business should be considered final.

Lesson Learnt - QA should be involved early in the project (V Model of testing)

As the software projects become more complex, it is very important that QA gets involved in it as early as requirements gathering.

Benefits from the V-model -

Defect Prevention

If QA gets involved in requirements discussions, defect prevention happens naturally.

• As requirements are being built, QA provides its feedback and helps in correcting mistakes in requirement gathering.
• QA walk thrus the requirement documents along with business stakeholders and point out technical or business issues based on prior test experiences.
• We see QA provide its comments on the solution on many occasions. Sometimes, extending an existing solution proves cost effective than starting a brand new solution.

Informed QA is a better tester 

The QA becomes more informed in the requirements gathering and subsequent project phases. A more informed QA will definitely prove to be a better tester in the future.

Saves Time and Learning

In a waterfall project, QA start test planning after the requirements document has been signed off. The activity of knowledge transfer is a very lengthy process. If the resource on boarded is not the same as the one in prior release,  the ramp up time could be very high. However if the QA was involved in requirements discussions, there is a good possibility that there would be a small or no learning curve.

QA and BA in sync

When QA starts test planning and design, the critical and high priority requirements have already been identified. At this stage, the QA lead already has a good idea of what his test strategy is going to be.

Relevant Links on web

http://en.wikipedia.org/wiki/V-Model_%28software_development%29

http://www.testingexcellence.com/v-model-in-software-testing/

Testing vs Quality Assurance

Popular Definitions

.

Quality Assurance (QA) is a way of preventing mistakes or defects in manufactured products and ISO 9000 defines it as "part of quality management focused on providing confidence that quality requirements will be fulfilled".  ( refer Wikipedia)

Quality Assurance is the prevention of faults by inspecting & testing.

 

Quality Control (QC) or Testing 

This defect prevention in quality assurance differs subtly from defect detection and rejection which happens in quality control.

Testing (or Quality Control) is detection of faults by actual sampling and inspection.

Related Web References

https://www.youtube.com/watch?v=KUASiRvGkeU

http://asq.org/pub/sqp/past/vol3_issue3/tian.html

Demand for software QA professionals soaring worldwide

As the software work understands the value of testing, the demand for software QA professionals continues to increase worldwide.

Cost of Quality

Cost of Quality = 
Cost of preventing defect +
Cost of appraisals +
Failure Costs

Defects can be prevented by walkthru of requirements and design. The appraisal cost is the cost spent on testing effort. The failure cost is the cost spent on fixing production tickets.

As can be seen, a defect becomes more expensive as it leaks into production. That is why, the failure costs are the highest.

The organizations have begun to realize that testing is not a cost or liability by itself. Instead, the investment in testing saves money in the long run when the software rolls into production. For this reason, as organizations make bigger and bigger investments in the information technology, the demand for QA professionals is also increasing.

Articles on the web

http://ianmartin.com/news/demand-for-qa-testers-soars-due-to-rising-software-usage/


http://www.computerworld.com/article/2487033/it-careers-tech-hotshots-the-rise-of-the-qa-expert.html

http://testhuddle.com/demand-for-qa-skills-on-the-rise/

 

Quality Organizations

Of the many organizations that are working to improve the testing processes and tools and also certify professionals with their testing knowledge, here are the two largest and most popular ones in USA.

ASQ - http://asq.org/index.aspx

iSTQB - http://www.istqb.org/

Quality and testing

What is Quality?

The Quality Processes

Before we get into quality assurance, we need to have a clear understanding of what is quality.

All projects have one or more objectives. The objectives are translated into requirements. Requirements could be functional or non functional.

Quality is the degree to which the end product meets the customer or user requirements.

What is Testing?

Testing is a set of activities to find defects in the end product. It is a process which helps to measure the divergence of the end product from the agreed requirements.

A defect is a finding where the end product does not follow a specific requirement.

Testing has evolved with time. Testing activities, tools and techniques have been defined based on the type of requirement. In order to find more defects in less time, the testing process and methodology continues to improve with new experiences.

History

The Indus Valley Civilization

Quality and testing have been prevalent in one form or the other right from the ancient history of mankind. If this was not true, we would not be able to see quality examples of application of  architecture, books, science, social principles and medicine even today.

Quality in Manufacturing

Application

The principles of quality and testing are being applied to projects in almost every field that we can think of - Software, construction, manufacturing, government, defense, medical sciences and retail. Interestingly, one can apply these principles even in the area of spirituality to measure one's progress on this path.