This version of SWEHB is associated with NPR 7150.2B. Click for the latest version of the SWEHB based on NPR7150.2C
3.12.6 The project manager shall define the milestones at which the software supplier(s) progress will be reviewed and audited as a part of the acquisition activities.
NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.
1.2 Applicability Across Classes
Classes F and G are labeled as "X (not OTS)" which means that the project is required to meet this requirement for all software that is not considered off-the-shelf.
Class A B C CSC D DSC E F G H Applicable?
Key: - Applicable | - Not Applicable
A & B = Always Safety Critical; C & D = Not Safety Critical; CSC & DSC = Safety Critical; E - H = Never Safety Critical.
For any software project, it is critical for management to review progress early and periodically to ensure the project remains on schedule, is progressing toward implementation of the requirements, and ultimately is addressing the customer's needs. It is also important for management to confirm periodically that the technical goals of the project are being achieved and that the technical direction of the project is appropriate (NPR 7123.1, NASA Systems Engineering Processes and Requirements). 041 Milestone reviews provide this type of management visibility into a project.
For software development that is acquired (supplied by a contractor), having regular progress reviews is even more important since these reviews are the keys to ensuring the contractor understood and will provide the product that NASA requested and that meets NASA's requirements for safety, quality, reliability, etc.
Milestone reviews can also serve to facilitate and ensure coordination between multiple development groups including development groups at multiple NASA Centers and contractors.
For acquired software development, milestone reviews are incorporated into the contract because the contract is the binding document for contractor performance and deliverables. Regardless of whether the development is in-house or contracted, the development agreement needs to contain, among other key elements, surveillance activities including monitoring activities, reviews, audits, decision points, meetings, known contract milestones, etc.
Other items related to milestone reviews to include in the development agreement are:
- Review periods for deliverables.
- Time period for making corrections to resolve findings.
- Formal reviews, such as those found in NPR 7123.1 041, NPR 7120.5 082, NPR 7120.7 264 (IT and Institutional Infrastructure) and NPR 7120.8 269(Research and Technology).
- Technical reviews.
- Progress reviews.
- Acceptance reviews.
Consult Center guidance on the following topics as Center guidance may provide insights into reviews and review topics to be considered for inclusion in the Statement of Work (SOW) and contract:
- Process Monitoring and Control (PMC).
See the 7.3 - Acquisition Guidance and 7.9 - Entrance and Exit Criteria topics in this Handbook for additional guidance on this topic. The references in 7.3 - Acquisition Guidance may provide additional guidance on project milestone reviews and topics for consideration. Topic 7.9 - Entrance and Exit Criteria describes inputs, material that will be reviewed, and outputs for each life-cycle milestone review which may be useful as input to the development of checklists for these reviews. Consult the NPR 7120 family of requirements documents for definitions of milestones and approaches for different project types.
Additional information and resources regarding software milestones are available in Software Processes Across NASA (SPAN), accessible to NASA users from the SPAN tab in this Handbook.
Keep in mind that reviews not included in the contract, may be difficult to require of the contractor, so it is important to ensure the SOW and other contract elements are reviewed by the proper project management and/or technical authority for completeness.
4. Small Projects
Project review plans and milestones are covered in NPR 7120.5 082, NPR 7120.7 264, NPR 7120.8 269, and NPR 7123.1 041. Projects are to ensure that software components are a part of specific project milestone reviews. Small projects need to determine a review process that meets the project requirements and contains adequate content to provide the greatest insights into progress toward the project's technical goals and the technical direction of the project.
- NASA Information Technology and Institutional Infrastructure Program and Project Management Requirements,NPR 7120.7, NASA Office of the Chief Engineer, 2008. The following appears on this document (web): SPECIAL ATTENTION: ONLY USE NID 7120.99: NASA Information Technology and Institutional Infrastructure Program and Project Management Requirements, as it is the interim directive to NPR 7120.7 and contains the most recent requirements. !DO NOT USE OTHER LINKS ON THIS PAGE!
Tools relative to this SWE may be found in the table below. You may wish to reference the Tools Table in this handbook for an evolving list of these and other tools in use at NASA. Note that this table should not be considered all-inclusive, nor is it an endorsement of any particular tool. Check with your Center to see what tools are available to facilitate compliance with this requirement.
No tools have been currently identified for this SWE. If you wish to suggest a tool, please leave a comment below.
6. Lessons Learned
A documented lesson from the NASA Lessons Learned database notes the following:
Acquisition and Oversight of Contracted Software Development (1999). Lesson Number 0921: Tailorable acquisition management and oversight processes for NASA contracted software development are essential to ensure that customers receive a quality product. A documented lesson from the NASA Lessons Learned database includes as a cause of the loss of a mission "the lack of a controlled and effective process for acquisition of contractor-developed, mission critical software." In this particular case, the quality of the contractor's product was not monitored as it would have been if the proper milestones for reviewing and auditing contractor progress were in place 528.