This version of SWEHB is associated with NPR 7150.2B. Click for the latest version of the SWEHB based on NPR7150.2D
View this section on the website
See edit history of this section
Post feedback on this section
1. Requirements
4.5.8 The project manager shall update software test plan(s) and software test procedure(s) to be consistent with software requirements.
1.1 Notes
NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.
1.2 Applicability Across Classes
Class A B C CSC D DSC E F G H Applicable?
Key: - Applicable | - Not Applicable
2. Rationale
Software test plans and test procedures are the main tools used to ensure proper implementation of the requirements and are developed based on those requirements. Therefore, if the requirements change, the test plans and procedures must also change to ensure that the test activity is accurate, complete, and consistent with the requirements.
Software test plans and test procedures are a key element of ensuring that the requirements that specify a product are completely and accurately implemented; in other words, that the delivered product is the right product.
3. Guidance
The team typically develops test plans and procedures as soon as the relevant phase in the life cycle has been completed. Once the initial documents have been created (see topic 7.18 - Documentation Guidance), it is important to keep them up-to-date as requirements change throughout the project life cycle. Continuous updates help avoid delays in testing caused by waiting until testing begins to update the test plans and procedures to accurately reflect the requirements.
Test documentation that may require updating includes:
- System test plan and procedures.
- Acceptance test plan and procedures.
- Unit test plans and procedures.
- Regression test plans and procedures.
- End-to-end test plans and procedures.
- Integration test plans and procedures.
- Test cases.
- Test scripts.
- Test data.
- Test schedule.
- Traceability matrix.
- Test effort estimates.
Using a traceability matrix which identifies the test plans, procedures, scripts, test cases and even test data associated with each requirement can be helpful when determining the effect of a requirements change on the testing documentation and plans. See SWE-072 for guidance regarding the traceability matrix for test plans.
It may be helpful to include as a checklist item in the relevant life cycle reviews confirmation that test documentation has been updated to reflect any requirements changes made at that point in the project life cycle. Additionally, if checklists for the test plans and procedures are used, consider including a checklist item to confirm that the plans and procedures are consistent with the software requirements; it may be helpful to repeat those checklists when making revisions to the test plans and procedures.
It may also be helpful if the project can establish a mechanism by which the test plan developers are notified of changes to the requirements once those changes are approved. Consider:
- Providing copies of approvals for change requests that affect requirements to the Software Lead Engineer.
- Providing copies of Change Control Board (CCB) minutes so test plan developers can check for approved requirements changes.
- Including a test team representative as part of the CCB or other authorized group responsible for approving changes to requirements.
When updating test plans and procedures, use configuration management principles (see SWE-080).
Consult Center Process Asset Libraries (PALs) for center-specific guidance and resources related to keeping test plans and procedures current as requirements change.
Additional guidance related to the test plans and procedures may be found in the following related requirements in this Handbook:
4. Small Projects
No additional guidance is available for small projects. The community of practice is encouraged to submit guidance candidates for this paragraph.
5. Resources
- (SWEREF-072) Checklist for the Contents of Software Critical Design Review (CDR), 580-CK-008-02, Software Engineering Division, NASA Goddard Space Flight Center (GSFC), 2010. This NASA-specific information and resource is available in Software Processes Across NASA (SPAN), accessible to NASA-users from the SPAN tab in this Handbook.
- (SWEREF-197) Software Processes Across NASA (SPAN) web site in NEN SPAN is a compendium of Processes, Procedures, Job Aids, Examples and other recommended best practices.
- (SWEREF-505) Public Lessons Learned Entry: 345.
5.1 Tools
Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN).
NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.
The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.
6. Lessons Learned
A documented lesson from the NASA Lessons Learned database notes the following:
- Mars Observer Attitude Control Fault Protection (Failure caused in part by incomplete testing.) Lesson Number 0345: "From the analyses performed after the Mars Observer [MO] mission failure, it became apparent that the MO fault protection suffered from a lack of top-down system engineering design approach. Most fault protection was in the category of low-level redundancy management. It was also determined that the MO fault protection software was never tested on the flight spacecraft before launch. Design fault protection to detect and respond to excessive attitude control errors, use RCS Thrusters to control excessive attitude control errors, and always test fault protection software on the flight spacecraft before launch." 505
