1. Requirement

4.5.9 The project manager shall provide and maintain bidirectional traceability from the software test procedures to the software requirements.

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 Applicability Across Classes

Key:    - Applicable | - Not Applicable

2. Rationale

Bidirectional traceability matrices help ensure that all of the software features and components contained in the software and specified in the requirements are tested. Bidirectional traceability also helps ensure that the software components and software requirements are tested and covered by a software test procedure.  

3. Guidance

Software test procedures are created to verify the software requirements for a project. In order to ensure that all requirements are verified via the test procedure set, the requirements need to be linked to the test procedures which verify them.

Traceability matrices help ensure that test procedures verify one or more software requirements by mapping those procedures back to one or more software requirements. Traceability is also used to ensure that the necessary level of test coverage is achieved, i.e., that there are sufficient tests to verify the requirements have been correctly implemented in the software.

Traceability links between individual requirements and other system elements, including, but not limited to, test procedures, are helpful tools when evaluating the impact of changing or deleting a requirement. When a requirement is changed, traceability can help identify the affected products, including design, documentation, source code, tests, etc. (NASA-GB-8719.13, NASA Software Safety Guidebook). ²⁷⁶

Traceability is important because it can point out software requirements that are not tested (i.e., missing tests) as well as tests that do not serve to test requirements (i.e., extra tests).

Bidirectional traceability is defined as a traceability chain that can be traced in both the forward and backward directions.  Figure 1 illustrates how software test procedures are traced between software products.

Before starting the traceability activity, it is assumed that the documents being traced (e.g., requirements, design, code, test data, etc.) have been approved.

Using a matrix such as the one shown below ³⁵⁶ allows a single exercise to show traceability both forwards and backwards. The matrix is completed left to right early in the appropriate phase in the project life cycle. As each column is completed, the forward trace is extended to the next set of products. Simply starting with a column, such as the LLD (low-level design) section and looking at the data in the columns to the left, shows the backward traceability from a LLD element to its parent HLD (high level design) element and back to the parent requirements.

While traceability matrices are not the only method for capturing bidirectional traceability, they are the most common. Traceability matrices can be included in the documents to which they apply, such as a test plan, or they can be combined into a single matrix covering higher level requirements, software requirements, design, code, and verification. General guidance for creating a bidirectional traceability matrix includes the following suggested actions:

• Create the matrix at the beginning of the project.
• Uniquely identify the elements in the matrix (requirements identifiers, design document identifiers and paragraph numbers, function identifiers, test identifiers, etc.).
• Keep the matrix maintained throughout the life of the project.
• Assign responsibility for creating and maintaining the matrix to a project team member, since managing the links/references can be a labor-intensive process that should be tracked/monitored.
• Maintain the matrix as an electronic document to make maintenance and reporting easier.
• Create the matrix such that it may be easily sorted to achieve/convey bi-directional traceability.
• Ensure a review of the matrix at major phases/key reviews of the project.

A bidirectional traceability matrix can be manually created and maintained, or may be a by-product of a requirements management tool. The tracing system needs to be chosen based on project complexity and the number of requirements. Check with project management to see if a requirements management tool exists for the local project that is capable of producing a bidirectional traceability matrix.

Keep in mind that a single requirement could trace to multiple test procedures. The reverse is also true, test procedures could trace back to multiple requirements, so the relationships identified in the matrix are not required to be one-to-one. The matrix needs to contain no missing relationships, i.e., empty cells in the matrix, as those indicate a problem with the set of test procedures which need to be designed such that every requirement is verified.

It is possible that new requirements may be generated during design and implementation.  When that happens and the requirements are confirmed as being within the scope of the project (not expanding the scope or “gold plating” the system by including unnecessary functionality), the traceability matrix needs to be revised to include the new requirements and the mapped design elements, implementation (source code), and test procedures.

To ensure full traceability between requirements and tests, it is important to trace test cases, test scripts, test data, and other supporting test information not already found in the test procedures to the relevant test procedures.  This level of trace information may or may not appear in a traceability matrix.  The test procedures, test cases, test scripts, and test data can include the proper links and references to ensure full traceability among all the elements of the tests.

Key aspects of tracing test procedures include:

• Ensure that tests for safety-critical functions are clearly identified either through the traceability matrix or through test procedure documentation.
• Trace each requirement and functional specification to one or more test cases (Manager's Handbook for Software Development ⁰³¹).If not already done, trace unit tests to source code and to design specifications.
• Trace integration tests to high-level design specifications.
• Trace system tests to software requirement specifications (SRS). ¹²⁷

NASA-specific traceability resources are available in Software Processes Across NASA (SPAN), accessible to NASA users from the SPAN tab in this Handbook. 

Additional guidance related to bidirectional traceability may be found in the following related requirements in this handbook:

4. Small Projects

For small projects without access to a requirements tool that includes tracing features and with time/budget limitations preventing them from acquiring a new tool and associated training, requirements tracing may be done with a spreadsheet (such as Microsoft ® Excel®), a simple database (such as Microsoft® Access®) or a textual document. It is very important that the project be diligent about keeping such traces up to date as these methods do not include automatic updates when requirements, design elements, or other relevant documents change.

Value-based requirement tracing may be an option for projects with small budgets where traceability of safety-critical requirements is the priority. "Value-based requirement tracing prioritizes all of the requirements in the system, with the amount of time and effort expended tracing each requirement depending on the priority of that requirement. This can save a significant amount of effort by focusing traceability activities on the most important requirements. However, value-based tracing requires a clear understanding of the importance of each requirement in the system; it may not be an option if full tracing is a requirement of the customer or the development process standards used for the project." (Kannenberg, 2009 ²³⁷)

5. Resources

5.1 Tools

Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN).

NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.

The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.

6. Lessons Learned

A documented lesson from the NASA Lessons Learned database notes the following:

Software Requirements Management. Lesson Number 3377: "The ability to manage and trace software requirements is critical to achieve success in any software project and to produce software products in a cost-effective and timely fashion. ⁵⁷⁶

• "Manual methods for management of software requirements are ineffective and inefficient, contributing to excessive costs as well as schedule delays. Aspects of the management of software requirements include the elicitation/specification, analysis, development, tracking, and changing of software requirements used during the implementation and sustaining phases of the software life cycle. Management and traceability of software requirements are critical to the success of producing reliable, high-quality, and safe software products that meet end-users requirements and needs in a cost-effective and timely fashion.
• "Cost and schedule impacts that result from incomplete, incorrect, or changing software requirements increase the later they occur in the software life cycle.
• "Current software technology, processes, and tools provide innovative automated methods to facilitate optimum management of software requirements (e.g., IBM Rational DOORS, IBM Rational RequisitePro, Cradle requirements management software).
• "Additionally, a collaborative relationship between the customer using the software and the developer providing the software is paramount to the success of the software project. More specifically, the users/customers must effectively define and accurately communicate their requirements to the developer. For example, the user's defined requirements should be clearly stated and unambiguous, concise, complete, autonomous, able to be implemented, and testable."