- 1. The Requirement
- 2. Rationale
- 3. Guidance
- 4. Small Projects
- 5. Resources
- 6. Lessons Learned
- 7. Software Assurance
4.6.4 The project manager shall complete, prior to delivery, verification that all software requirements identified for this delivery have been met, that all approved changes have been implemented, and that all defects designated for resolution prior to delivery have been resolved.
NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.
Click here to view the history of this requirement: SWE-194 History
1.3 Applicability Across Classes
Key: - Applicable | - Not Applicable
A & B = Always Safety Critical; C & D = Sometimes Safety Critical; E - F = Never Safety Critical.
Requirements are the basis for building a product that meets the needs of the project. They define the behavior of the system and the constraints under which the problem is to be solved. It is important that the implemented product is verified against the requirements to show that it actually meets all of the requirements.
However, requirements change over time and the documentation of the requirements is not always up to date. Also, for Agile approaches, not all of the requirements are necessarily documented upfront. It is critical that the Project Manager uses an appropriate method or tool to keep track of the requirements necessary for each delivery of the software and that these are kept up to date with any changes or defect resolutions.
The verification planning, execution, and results should include all requirements for that delivery. A traceability matrix that shows all of the planned and modified requirements to the test procedures/results is critical for showing compliance to this NPR requirement. Also critical are the test reports that show that each requirement verification passed.
This requirement applies to all NASA centers and Software Classes A, B, C, D, and F.
For the non-Agile lifecycle, an analysis of the baseline set of requirements and all change requests would need to be performed against the verification plans, procedures, and results to show that every requirement has been verified successfully.
For an Agile lifecycle, for each planned release, an analysis of the set of requirements associated with that release and all change requests associated with those requirements would need to be performed against the verification plans, procedures, and results to show that every requirement has been verified successfully. Additionally, any defect resolutions that were designated for inclusion in the release need to be analyzed against the verification procedures and results to ensure that they were resolved and successfully verified.
Keeping track of requirements changes can be done by several methods as described in SWE-053.
Verification is done by several methods. For guidance on inspections, see section 5.3 in the NPR. For guidance on testing, see section 4.5 in the NPR, specifically SWE-065, 066, 068, 071, and 191.
For this effort, having a complete bi-directionality between requirements, design, and verification is significant and will make the analysis easier.
4. Small Projects
No additional guidance is available for small projects.
No references have been currently identified for this SWE. If you wish to suggest a reference, please leave a comment below.
6. Lessons Learned
6.1 NASA Lessons Learned
No Lessons Learned have currently been identified for this requirement.
6.2 Other Lessons Learned
No other Lessons Learned have currently been identified for this requirement.
7. Software Assurance
7.1 Tasking for Software Assurance
Confirm that the project has identified the software requirements to be met, the approved changes to be implemented, and defects to be resolved for each delivery.
Confirm that the project has met all software requirements identified for the delivery.
Confirm that approved changes have been implemented and tested.
Confirm that the approved changes to be implemented and the defects to be resolved have been resolved.
Approve or sign-off on the projects delivered products.
7.2 Software Assurance Products
- Evidence that SA has approved or signed-off on the delivered products.
- Evidence that delivery tasks 1 through 4 have been performed.
List of any risks and issues found with delivery.
- Problem/change report status: total number, number closed, number opened in the current reporting period, age, severity.
- Assessment of the software test coverage: a measure used to describe the degree to which the source code of a project is tested by a particular test suite.
- Percentage complete of Software Requirements to Test Procedures; defect data.
For each delivery, review the delivery documentation and confirm that the document identifies the particular requirements that have been implemented for this delivery, any changes that are approved for implementation in this delivery and any defects that need to be resolved/implemented before delivery. It is usually helpful for the project to include a list of defects or discrepancies that have not been resolved in this delivery and will either need to be resolved in a future delivery or in maintenance. Delivery documentation for this information may include a delivery letter and a version description document.
The configuration management records for any changes, corrections or new capabilities/requirements should indicate which of the change requests have been approved and the planned delivery for their implementation. Confirm that all the requirements identified for this delivery have been successfully tested by reviewing the traceability matrices to see which tests map to the requirements for the delivery and checking the test reports for those requirements.
Review the set of approved changes for the software to be delivered and confirm that all of the approved changes that were scheduled to be implemented by delivery are actually in the software deliverable. Also, confirm that:
- The changes have been tested to confirm the approved change produces correct results,
- If the change was a defect, the changes fix the defect, and that checks have been made to catch any similar defects elsewhere in the deliverable software,
- Regression tests have been successfully run to ensure previously tested capabilities of the software still work. A full set of regression tests should be run on critical software.
For the set of defects identified in the software to be delivered, confirm that all defects planned for delivery have been resolved in one of the following ways:
- The defect is fixed and tested (confirming the above)
- The defect has an agreed-upon work-around (documented in the user documentation)
- The defect has been discussed with the customer and agreement was reached to defer the defect to a later build or maintenance or to leave the defect as is. If the agreement is to leave the defect “as is”, identify any risk associated with this decision
Software assurance personnel should sign off on the delivery documentation signifying that the delivery is complete and correct
For guidance on testing, see section 4.5 in the NPR, specifically SWE-065, 066, 068, 071, and 191. For more guidance on managing changes and release management, see SWE-053, 080 and 085.