Unknown macro: {div3}

1. Requirements

2.5.6 The project shall define the milestones at which the software supplier(s) progress will be reviewed and audited as a part of the acquisition activities.

1.1 Notes

Known contract milestones are expected to be included in the resulting contract.

1.2 Applicability Across Classes

Class D not Safety Critical, and Class G are labeled with "P(Center)". This means that an approved Center-defined process that meets a non-empty subset of the full requirement can be used to achieve this requirement.

Class F is labeled as "X(not OTS)" which means that the project is required to meet this requirement for all software that is not considered off-the-shelf.

Class	A_SC	A_NSC	B_SC	B_NSC	C_SC	C_NSC	D_SC	D_NSC	E_SC	E_NSC	F	G	H
Applicable?								P(C)			X	P(C)

Key:    A_SC = Class A Software, Safety-Critical | A_NSC = Class A Software, Not Safety-Critical | ... | - Applicable | - Not Applicable
^X - Applicable with details, read above for more | P(C) - P(Center), follow center requirements or procedures

Unknown macro: {div3}

2. Rationale

For any software project, it is critical for management to review progress early and periodically to ensure the project remains on schedule, is progressing toward implementation of the requirements, and ultimately is addressing the customer's needs. It is also important for management to confirm periodically that the technical goals of the project are being achieved and that the technical direction of the project is appropriate (NASA Systems Engineering Processes and Requirements, NPR 7123.1). Milestone reviews provide this type of management visibility into a project.

For software development that is acquired (supplied by a contractor), having regular progress reviews is even more important since these reviews are the keys to ensuring the contractor understood and will provide the product that NASA requested and that meets NASA's requirements for safety, quality, reliability, etc.

Milestone reviews can also serve to facilitate and ensure coordination between multiple development groups including development groups at multiple NASA Centers and contractors.

Unknown macro: {div3}

3. Guidance

For acquired software development, milestone reviews should be incorporated into the contract because the contract is the binding document for contractor performance and deliverables. The contract should contain, among other key elements, surveillance activities including monitoring activities, reviews, audits, decision points, meetings, etc.

Other items related to milestone reviews that should be included in the contract are:

• Review periods for deliverables
• Time period for making corrections to resolve findings
• Formal reviews, such as those found in NPR 7123.1A, NPR 7120.5D, NPR 7120.7 (IT and Institutional Infrastructure) and NPR 7120.8 (Research and Technology)
• Technical reviews
• Progress reviews
• Acceptance reviews

Consult Center guidance on the following topics as this guidance may provide insights into reviews and review topics that should be considered for inclusion in the SOW and contract:

• Acquisition
• Planning
• Scheduling
• Process Monitoring and Control (PMC)

See the ?[7.7 - Acquisition Guidance] and [7.3 - Entrance and Exit Criteria] topics in this handbook for additional guidance on this topic. The references in [7.7 - Acquisition Guidance] may provide additional guidance on project milestone reviews and topics for consideration. Topic [7.3 - Entrance and Exit Criteria] describes inputs, material that will be reviewed, and outputs for each life cycle milestone review which may be useful as input to the development of checklists for these reviews.
 
Keep in mind that reviews not included in the contract, may be difficult to require of the contractor, so it is important to ensure the Statement of Work (

<ac:macro ac:name="unmigrated-wiki-markup">
<ac:plain-text-body><![CDATA[

SOW

]]></ac:plain-text-body>
</ac:macro>

) and other contract elements are reviewed by the proper project management and/or technical authority for completeness.

Unknown macro: {div3}

4. Small Projects

For projects designated "small" by center criteria or designate as a high risk via NASA payload risk classifications (NPR 8705.4), it may be possible to reduce the number of reviews to meet time and cost constraints. Keep in mind, however, that milestone reviews should not be eliminated, as these reviews are critical checkpoints in the life cycle of the project. Projects covered by NPR 7120 and NPR 7123.1A are required to follow specific project milestone reviews with software components. Small projects should determine the set of reviews that provide the greatest insights into progress toward the project's technical goals and the technical direction of the project.

Unknown macro: {div3}

5. Resources

1. NASA Procedural Requirement, NASA Systems Engineering Processes and Requirements w/Change 1, NPR 7123.1A, 2009.
2. NASA Procedural Requirement, NASA Space Flight Program and Project Management Requirements, NPR 7120.5D (NM 7120-81), 2010.
3. ?[7.7 - Acquisition Guidance]
4. ?[7.3 - Entrance and Exit Criteria]
5. Project Monitoring & Control (PMC), Goddard Space Flight Center (GSFC) Process Asset Library (PAL)

5.1 Tools

Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN).

NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.

The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.

Unknown macro: {div3}

6. Lessons Learned

Tailorable acquisition management and oversight processes for NASA contracted software development are essential to ensure that customers receive a quality product. A documented lesson from the NASA Lessons Learned database includes as a cause of the loss of a mission "the lack of a controlled and effective process for acquisition of contractor-developed, mission critical software." In this particular case, the quality of the contractor's product was not monitored as it would have been if the proper milestones for reviewing and auditing contractor progress were in place. (http://www.nasa.gov/offices/oce/llis/0921.html)