bannerb

This version of SWEHB is associated with NPR 7150.2B. Click for the latest version of the SWEHB based on NPR7150.2C

SWE-072 - Bidirectional Traceability Between Software Test Procedures and Software Requirements

1. Requirement

4.5.9 The project manager shall provide and maintain bidirectional traceability from the software test procedures to the software requirements.

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 Applicability Across Classes

Class

     A      

     B      

     C      

   CSC   

     D      

   DSC   

     E      

     F      

     G      

     H      

Applicable?

   

   

   

   

   

   

   

   

   

   

Key:    - Applicable | - Not Applicable
A & B = Always Safety Critical; C & D = Not Safety Critical; CSC & DSC = Safety Critical; E - H = Never Safety Critical.


2. Rationale

Bidirectional traceability matrices help ensure that all of the software features and components contained in the software and specified in the requirements are tested. Bidirectional traceability also helps ensure that the software components and software requirements are tested and covered by a software test procedure.  

3. Guidance

Software test procedures are created to verify the software requirements for a project. In order to ensure that all requirements are verified via the test procedure set, the requirements need to be linked to the test procedures which verify them.

Traceability matrices help ensure that test procedures verify one or more software requirements by mapping those procedures back to one or more software requirements. Traceability is also used to ensure that the necessary level of test coverage is achieved, i.e., that there are sufficient tests to verify the requirements have been correctly implemented in the software.

Traceability links between individual requirements and other system elements, including, but not limited to, test procedures, are helpful tools when evaluating the impact of changing or deleting a requirement. When a requirement is changed, traceability can help identify the affected products, including design, documentation, source code, tests, etc. (NASA-GB-8719.13, NASA Software Safety Guidebook). 276

Traceability is important because it can point out software requirements that are not tested (i.e., missing tests) as well as tests that do not serve to test requirements (i.e., extra tests).

Bidirectional traceability is defined as a traceability chain that can be traced in both the forward and backward directions.  Figure 1 illustrates how software test procedures are traced between software products.

Before starting the traceability activity, it is assumed that the documents being traced (e.g., requirements, design, code, test data, etc.) have been approved.

Using a matrix such as the one shown below 356 allows a single exercise to show traceability both forwards and backwards. The matrix is completed left to right early in the appropriate phase in the project life cycle. As each column is completed, the forward trace is extended to the next set of products. Simply starting with a column, such as the LLD (low-level design) section and looking at the data in the columns to the left, shows the backward traceability from a LLD element to its parent HLD (high level design) element and back to the parent requirements.

While traceability matrices are not the only method for capturing bidirectional traceability, they are the most common. Traceability matrices can be included in the documents to which they apply, such as a test plan, or they can be combined into a single matrix covering higher level requirements, software requirements, design, code, and verification. General guidance for creating a bidirectional traceability matrix includes the following suggested actions:

  • Create the matrix at the beginning of the project.
  • Uniquely identify the elements in the matrix (requirements identifiers, design document identifiers and paragraph numbers, function identifiers, test identifiers, etc.).
  • Keep the matrix maintained throughout the life of the project.
  • Assign responsibility for creating and maintaining the matrix to a project team member, since managing the links/references can be a labor-intensive process that should be tracked/monitored.
  • Maintain the matrix as an electronic document to make maintenance and reporting easier.
  • Create the matrix such that it may be easily sorted to achieve/convey bi-directional traceability.
  • Ensure a review of the matrix at major phases/key reviews of the project.

A bidirectional traceability matrix can be manually created and maintained, or may be a by-product of a requirements management tool. The tracing system needs to be chosen based on project complexity and the number of requirements. Check with project management to see if a requirements management tool exists for the local project that is capable of producing a bidirectional traceability matrix.

Keep in mind that a single requirement could trace to multiple test procedures. The reverse is also true, test procedures could trace back to multiple requirements, so the relationships identified in the matrix are not required to be one-to-one. The matrix needs to contain no missing relationships, i.e., empty cells in the matrix, as those indicate a problem with the set of test procedures which need to be designed such that every requirement is verified.


It is possible that new requirements may be generated during design and implementation.  When that happens and the requirements are confirmed as being within the scope of the project (not expanding the scope or “gold plating” the system by including unnecessary functionality), the traceability matrix needs to be revised to include the new requirements and the mapped design elements, implementation (source code), and test procedures.

If the software verification team is not the same as the requirements development team, collaboration may be needed to ensure proper bidirectional traceability between test procedures and requirements.

To ensure full traceability between requirements and tests, it is important to trace test cases, test scripts, test data, and other supporting test information not already found in the test procedures to the relevant test procedures.  This level of trace information may or may not appear in a traceability matrix.  The test procedures, test cases, test scripts, and test data can include the proper links and references to ensure full traceability among all the elements of the tests.

Key aspects of tracing test procedures include:

  • Ensure that tests for safety-critical functions are clearly identified either through the traceability matrix or through test procedure documentation.
  • Trace each requirement and functional specification to one or more test cases (Manager's Handbook for Software Development 031).If not already done, trace unit tests to source code and to design specifications.
  • Trace integration tests to high-level design specifications.
  • Trace system tests to software requirement specifications (SRS). 127

NASA-specific traceability resources are available in Software Processes Across NASA (SPAN), accessible to NASA users from the SPAN tab in this Handbook. 

Additional guidance related to bidirectional traceability may be found in the following related requirements in this handbook:

SWE-052

Bidirectional Traceability Between Higher Level Requirements and Software Requirements

SWE-059

Bidirectional Traceability Between Software Requirements and Software Design

SWE-064

Bidirectional Traceability Between Software Design and Software Code

4. Small Projects

For small projects without access to a requirements tool that includes tracing features and with time/budget limitations preventing them from acquiring a new tool and associated training, requirements tracing may be done with a spreadsheet (such as Microsoft ® Excel®), a simple database (such as Microsoft® Access®) or a textual document. It is very important that the project be diligent about keeping such traces up to date as these methods do not include automatic updates when requirements, design elements, or other relevant documents change.

Value-based requirement tracing may be an option for projects with small budgets where traceability of safety-critical requirements is the priority. "Value-based requirement tracing prioritizes all of the requirements in the system, with the amount of time and effort expended tracing each requirement depending on the priority of that requirement. This can save a significant amount of effort by focusing traceability activities on the most important requirements. However, value-based tracing requires a clear understanding of the importance of each requirement in the system; it may not be an option if full tracing is a requirement of the customer or the development process standards used for the project." (Kannenberg, 2009 237)

5. Resources


5.1 Tools

Tools relative to this SWE may be found in the table below. You may wish to reference the Tools Table in this handbook for an evolving list of these and other tools in use at NASA. Note that this table should not be considered all-inclusive, nor is it an endorsement of any particular tool. Check with your Center to see what tools are available to facilitate compliance with this requirement.

Tool nameTypeOwner/SourceLinkDescriptionUser

Requirements Experts

COTS

Requirements Experts

https://reqexperts.com/resources/tools-templates/ ...

Tools and Templates for Developing Requirements. Others services and training also available.

Requisite Pro

COTS

IBM Rational

https://www.ibm.com/support/knowledgecenter/en/SSSHCT7.1.0/com.ibm.reqpro.help/getstart/cproductover ...

Rational RequisitePro helps project teams to manage their requirements, to write good use cases, to improve traceability, to strengthen collaboration, to reduce project rework, and to increase quality. Version 7.1.0.

KSC

RequirementsLink

COTS

ENSER/Parametric Technology Corporation (PTC)

http://www.ptc.com/appserver/wcms/relnotes/note.jsp?icgdbkey=826imdbkey=119829 ...

Windchill RequirementsLink. Requirements capture and tracking tool. Windchill RequirementsLink - an integral option for Windchill PDMLink - lets you manage product requirements, including change control and associating requirements with specific product structures and design content. With bi-directional traceability between customer needs, market requirements and the underlying technical requirements, you can ensure that customer and market requirements are satisfied by designs, and properly verified during development.

SSC

Requirements traceability tool

SPAN - Accessible to NASA users via SPAN tab in this Handbook. By Request - Non-NASA users, contact User for a copy of this tool.

GSFC

http://software.gsfc.nasa.gov/tools/RequirementsTool080905.xls ...

This tool, an Excel spreadsheet, provides bidirectional traceability between requirements, design, code, and test procedures. For a list of features, see the tools section of the GSFC PAL. Note: You must be on site at Goddard in order to access this file. Search in span for "GSFC_TL_20161114_Req_Trace_Tool"

GSFC

PTC Integrity

COTS

PTC

http://www.ptc.com/application-lifecycle-management/integrity/lifecycle-manager ...

PTC Integrity is a systems and software lifecycle management (SSLM) and application lifecycle management (ALM) platform used for Process automation and workflow management

IV&V, GSFC

DOORS®

COTS

IBM® Rational®

http://www-01.ibm.com/software/awdtools/doors/ ...

IBM® Rational® DOORS® family is a group of requirements management tools that allow you to capture, trace, analyze and manage changes across the development lifecycle.

ARC, DFRC, GRC, GSFC, IV&V, JPL, JSC, JSC, LaRC, MSFC,

Dimensions® RM

COTS

MicroFocus

https://www.microfocus.com/products/dimensions-rm/ ...

Create and manage requirements more efficiently. Dimensions® RM (formerly Serena® Dimensions RM) increases visibility and collaboration across business and delivery teams. Powerful reporting and tracking provide end-to-end traceability from initial concepts to production delivery.

Caliber®

COTS

Micro Focus

https://www.microfocus.com/products/requirements-management/caliber/ ...

CaliberRM® is a requirements management tool that ensures that applications meet end user needs. Using CaliberRM, analysts, developers, testers and other stakeholders accurately capture and communicate the user's requirements throughout the application lifecycle. Version 11.5.

A bidirectional traceability matrix can be manually created and maintained, or may be a by-product of a requirements management tool. The tracing system needs to be chosen based on project complexity and the number of requirements. Check with project management to see if a requirements management tool exists for the local project that is capable of producing a bidirectional traceability matrix.

6. Lessons Learned

A documented lesson from the NASA Lessons Learned database notes the following:

Software Requirements Management. Lesson Number 3377: "The ability to manage and trace software requirements is critical to achieve success in any software project and to produce software products in a cost-effective and timely fashion. 576

  • "Manual methods for management of software requirements are ineffective and inefficient, contributing to excessive costs as well as schedule delays. Aspects of the management of software requirements include the elicitation/specification, analysis, development, tracking, and changing of software requirements used during the implementation and sustaining phases of the software life cycle. Management and traceability of software requirements are critical to the success of producing reliable, high-quality, and safe software products that meet end-users requirements and needs in a cost-effective and timely fashion.
  • "Cost and schedule impacts that result from incomplete, incorrect, or changing software requirements increase the later they occur in the software life cycle.
  • "Current software technology, processes, and tools provide innovative automated methods to facilitate optimum management of software requirements (e.g., IBM Rational DOORS, IBM Rational RequisitePro, Cradle requirements management software).
  • "Additionally, a collaborative relationship between the customer using the software and the developer providing the software is paramount to the success of the software project. More specifically, the users/customers must effectively define and accurately communicate their requirements to the developer. For example, the user's defined requirements should be clearly stated and unambiguous, concise, complete, autonomous, able to be implemented, and testable."
  • No labels