1. Requirements

3.1.7 The project manager shall define and document the milestones at which the software developer(s) progress will be reviewed and audited. 

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 History

1.3 Applicability Across Classes

Classes F and G are labeled as "X (not OTS)" which means that the project is required to meet this requirement for all software that is not considered off-the-shelf.

Key:    - Applicable | - Not Applicable

2. Rationale

For any software project, it is critical for management to review progress early and periodically to ensure the project remains on schedule, is progressing toward implementation of the requirements, and ultimately is addressing the customer's needs. It is also important for management to confirm periodically that the technical goals of the project are being achieved and that the technical direction of the project is appropriate (NPR 7123.1, NASA Systems Engineering Processes and Requirements). ⁰⁴¹ Milestone reviews provide this type of management visibility into a project.

For software development that is acquired (supplied by a contractor), having regular progress reviews is even more important since these reviews are the keys to ensuring the contractor understood and will provide the product that NASA requested and that meets NASA's requirements for safety, quality, reliability, etc.

Milestone reviews can also serve to facilitate and ensure coordination between multiple development groups including development groups at multiple NASA Centers and contractors.

This requirement ensures that the software development effort is systematically monitored and evaluated to verify progress, detect issues early, and ensure alignment with project objectives, mission requirements, stakeholder expectations, and safety standards. Defining and documenting review and audit milestones provides a clear structure that promotes accountability, transparency, and compliance throughout the software development life cycle. The rationale for this requirement is explained below:

1. Ensures Progress Tracking

• Why It Matters:
  • Defined milestones provide discrete points in the project where the software team's progress can be reviewed objectively. These checkpoints help prevent delays or deviations from the planned schedule.
  • Without milestones, project managers may lose visibility into the development timeline, increasing the risk of missed deadlines or compromised deliverables.
• Rationale:
  • Milestones enable periodic evaluation of progress and allow proactive measures if development falls behind schedule or strays from planned objectives.

2. Validates Deliverables

• Why It Matters:
  • Auditing development progress at predefined milestones ensures that intermediate artifacts and deliverables (e.g., specifications, designs, test cases, prototypes, etc.) meet established acceptance criteria before development proceeds.
  • Unverified or unchecked deliverables may result in defects propagating downstream, increasing the risk of rework and compromising overall quality.
• Rationale:
  • Milestone reviews ensure that deliverables are validated against acceptance criteria, minimizing the risk of issues in subsequent phases.

3. Provides Early Detection of Issues

• Why It Matters:
  • Regular reviews act as a checkpoint for identifying problems such as technical challenges, resource bottlenecks, or compliance gaps before they escalate and impact the project’s objectives.
  • Without defined milestones, issues may go unnoticed, leading to significant delays, budget overruns, or mission-critical failures.
• Rationale:
  • Milestones serve as early detection mechanisms to identify risks or deficiencies, enabling corrective actions to be taken before they jeopardize the project.

4. Promotes Accountability

• Why It Matters:
  • Milestones create clear expectations for progress and deliverables, ensuring accountability for the software development team, assurance personnel, and project leadership.
  • Without milestones, responsibilities may become unclear, leading to inefficiencies in team collaboration and a lack of visibility into individual contributions.
• Rationale:
  • Documenting review points fosters accountability among all stakeholders and ensures that each phase progresses according to the agreed schedule and requirements.

5. Aligns Teams Toward a Common Goal

• Why It Matters:
  • Milestone reviews facilitate communication between stakeholders, including developers, assurance personnel, customers, and the project manager. They serve as a forum to verify that all teams are aligned with the project’s goals and requirements.
  • Misalignment among teams can result in defects, unnecessary rework, or delays in integration.
• Rationale:
  • Milestones ensure consistent collaboration and alignment of priorities across all parties involved in the software development process.

6. Establishes Formal Verification and Compliance Checkpoints

• Why It Matters:
  • Aerospace software development is subject to strict compliance standards (e.g., NPR 7150.2, DO-178C, NASA systems engineering requirements). Milestone reviews act as formal audits that verify adherence to these standards.
  • Failure to verify compliance during the project lifecycle can result in major penalties, rejection of deliverables, or even mission failure.
• Rationale:
  • Milestones provide formal compliance checkpoints, ensuring regulatory requirements and quality standards are continuously upheld.

7. Facilitates Risk Management

• Why It Matters:
  • Regular audits and reviews reduce uncertainty by uncovering risks related to technical failures, deviations from project scope, resource constraints, or integration issues.
  • Without milestones, risks may accumulate or remain unidentified until late in the project lifecycle, leading to costly and time-consuming corrections.
• Rationale:
  • Establishing milestones helps mitigate risks by allowing project teams to evaluate and address vulnerabilities at predefined intervals.

8. Enables Controlled Iterative Development

• Why It Matters:
  • Iterative approaches, especially in agile or phased software development, benefit from milestones that confirm progress before moving on to subsequent phases. Milestones ensure that iterations meet specific objectives before additional features or integrations are added.
  • Without controlled iteration, development efforts may result in wasted efforts, abandoned code, or insufficient system validation.
• Rationale:
  • Milestones structure iterative development, ensuring each phase delivers the foundation necessary for subsequent work.

9. Supports Efficient Resource Allocation

• Why It Matters:
  • Milestones provide opportunities to reassess resource allocation based on project progress and identified risks. If problems arise during a review, adjustments to manpower, budget, or tools can be made accordingly.
  • Unchecked progress may lead to over- or underutilization of resources, increasing inefficiencies.
• Rationale:
  • Documenting milestone audits allows the project manager to make timely resource adjustments, ensuring that development proceeds efficiently.

10. Improves Transparency for Stakeholders

• Why It Matters:
  • Clients, customers, and stakeholders benefit from visibility into project progress. Milestones provide key reporting points where stakeholders can review deliverables, approve outcomes, and provide feedback.
  • Lack of visibility can lead to stakeholder dissatisfaction, misaligned expectations, or delayed approvals.
• Rationale:
  • Milestones improve communication and transparency, keeping stakeholders informed and engaged throughout the development lifecycle.

11. Supports Certification and Approval

• Why It Matters:
  • Aerospace software often requires formal audits and certifications (e.g., flight readiness reviews, safety certifications). Milestones enable structured audits that provide evidence of the software's reliability and readiness for certification.
  • Late certification reviews without milestone-based evaluations risk missing essential verifications or delaying approvals.
• Rationale:
  • Milestones support systematic certification processes by ensuring the software undergoes necessary reviews and audits prior to acceptance.

12. Establishes Exit and Entrance Criteria

• Why It Matters:
  • Milestones serve as gating mechanisms that establish exit criteria for progress to the next phase and entrance criteria for subsequent activities. This ensures work is complete and verified before proceeding.
  • Lack of exit and entrance criteria risks incomplete preparation for downstream tasks or phases, leading to integration or testing complications.
• Rationale:
  • Defining milestones ensures prerequisites are satisfied before entering critical phases, reducing risks of downstream defects or delays.

Implementation Notes

• Milestones will vary depending on project complexity, scope, and lifecycle model (e.g., waterfall, agile, hybrid). For instance:
  • In waterfall projects, milestones typically align with phased delivery (e.g., requirements review, design review, testing phases).
  • In agile projects, milestones can include sprint reviews, backlog refinements, or customer acceptance points for iterative deliverables.
• Common milestone review types include:
  • Requirements Review: Ensures customer needs are fully defined and understood.
  • Preliminary Design Review (PDR): Verifies technical feasibility and compliance with system requirements.
  • Critical Design Review (CDR): Confirms the design is ready for implementation.
  • Test Readiness Review (TRR): Validates testing plans and resources.
  • System Integration Review (SIR): Ensures proper system-level integration and interoperability.
  • Delivery Review: Documents readiness for approval and handover.

Conclusion

Defining and documenting milestones for progress reviews and audits ensures the project manager can maintain oversight, provide consistent progress evaluation, detect risks early, and maintain compliance with regulatory standards. Milestones improve collaboration, align stakeholder priorities, and establish clear checkpoints for all phases of development, promoting efficiency, accountability, and quality assurance. These structured review points are essential for managing risk and ensuring mission success in aerospace software projects.

3. Guidance

The purpose of the milestone review meeting is to review the overall milestone progress and accomplishments at selected milestones/phases during the project life cycle. Milestone reviews are generally defined by the project manager. This meeting also facilitates the project team to have a get-together with higher management and other stakeholders and to discuss project issues; share lessons learned and suggest improvements for the next milestones.

The milestone review guidance provides refined explanations to highlight the importance of milestones, detailed best practices for selecting and managing them, and practical steps for using milestones effectively. The aim is to promote clarity, consistency, and efficiency in managing milestone progress reviews across all types of projects, including acquired software development.

Purpose of the Milestone Review Meeting

The milestone review meeting serves to evaluate the project's progress and accomplishments at predetermined points during the software development life cycle. It is an opportunity to ensure alignment with project goals, address issues, and optimize plans for future work. Milestone reviews also enable the project manager and team to engage with higher-level management and other stakeholders to review performance, suggest improvements, and share lessons learned.

3.1 Why Milestones Matter

Milestones are an integral part of project planning and execution. They serve as checkpoints to measure progress, ensure alignment with the overall schedule, and enable course corrections if needed. Proper use of milestone reviews enhances clarity, accountability, and transparency across all phases of the project.

Key Benefits of Milestones

1. Progress Tracking:

   • Milestones provide clear, measurable points to compare actual progress with planned objectives. For smaller or simpler projects, milestones may be limited to critical points like project start and end dates.
   • In long-duration projects, milestones act as checkpoints to ensure steady progress. For example:
     • At 25% of the project timeline, approximately 25% of the deliverables should be complete, and the review evaluates whether this is on track.

2. Early Problem Detection:

   • Milestone reviews offer an early warning system. For instance, if the 25% milestone shows only 15% completion, the discrepancy signals potential risk and highlights areas that need immediate attention.

3. Success Communication:

   • Milestones provide a clear way to showcase accomplishments and build stakeholder trust. By communicating progress at each milestone, stakeholders remain updated on the project's status and outcomes.
   • This is especially useful in complex or mission-critical projects, where the timely completion of interim steps is vital.

Milestones are less common in Agile projects due to the flexibility of Agile methodologies, which focus on iterations rather than predefined phases. For Agile workflows, milestones are often tied to the end of sprints or major objectives, serving as reflections of progress over time rather than strict deadlines.

3.2 How to Select Milestones

Criteria for Identifying Milestones

When determining appropriate milestones, the following principles and questions can help ensure the right balance between too few and too many milestones:

1. Essential Practice: Select milestones tied to meaningful, high-impact points in the project, avoiding excessive or inconsequential milestones.
2. Clear Definition: Milestones must represent well-defined events, rather than activities that require time (e.g., completion of a design review, approval of requirements, delivery of a code base).
3. Stakeholder Relevance: Choose milestones that demand review and approval from stakeholders.

Guiding Questions to Define Milestones:

• Is the milestone associated with a key deliverable or end product?
• Is this milestone time-sensitive (e.g., will missing it impact final deadlines)?
• Is this milestone critical to overall project success or indicative of significant progress?
• Does the milestone require review and sign-off by management, stakeholders, or customers?
• Does the milestone represent an input dependency from an external entity (e.g., subcontractors or vendors)?

Example Milestone Selection for Software Projects:

• Software requirements review.
• Preliminary software design completion.
• Delivery of an initial build (prototyping).
• Completion of integration testing.
• Deployment readiness review.

For External Dependencies:

• Subcontractor deliverables (e.g., materials, prototypes, or sub-assemblies).
• Critical delivery dates for items required to meet project deadlines.

3.3 How to Use Milestones in Project Management Software

Software tools can significantly enhance milestone tracking by linking tasks, deliverables, and progress to key checkpoints. Effective use of project management software allows for:

1. Task-Milestone Organization:

   • Connect tasks to associated milestones to provide clarity on how individual contributions impact broader goals.
   • Example: For a software update, major milestones could be:
     • Finalizing software design → Design-related tasks mapped to this milestone.
     • Completing production → Assign production tasks.
     • Testing completeness → Link testing tasks.

2. Defining Check-in Points:

   • Milestone reviews should be scheduled consistently with overall project goals. Key tasks and expectations for each milestone should be clearly defined.

3. Progress Visualization:

   • Use milestone tracking features to provide a graphical or report-based view of overall project progress.
   • Notify team members when milestone deadlines or review meetings are approaching.

3.4 Tasks Associated with Milestone Reviews

Milestone reviews involve detailed preparation, organization, and follow-up actions. Below are the recommended steps for conducting these reviews effectively:

Preparation Phase

1. The project manager calculates milestone progress compared to planned deliverables and prepares a milestone review report based on established templates.
2. Send the agenda, review report, and meeting invitation to all relevant stakeholders as scheduled.

During the Review

1. Conduct milestone review meetings:
   • Assess progress and overall milestone status.
   • Discuss and resolve open issues.
   • Record lessons learned and improvement suggestions.
2. Document and log action items and issues into an issue management system.

Follow-Up Phase

1. Distribute the milestone review report, including meeting outcomes, to stakeholders and customer contacts.
2. Assign action items to team members or stakeholders, ensuring clarity of their responsibilities.
3. Track all action items to closure and provide regular updates on their resolution.

3.5 Acquired Software Development and Milestone Reviews

For acquired software development, milestone reviews must be explicitly incorporated into the contract to ensure compliance and enforceability. The contract serves as the binding document for defining:

• Contractor responsibilities.
• Deliverable review and approval timelines.
• Monitoring activities (e.g., audits, technical reviews, and progress reports).

Key Considerations for Acquired Software Projects

• Include review periods for contractor deliverables to facilitate approval and revisions.
• Define entrance and exit criteria for reviews as specified in NPR 7123.1, NPR 7120.5, and related documents. See 7.09 - Entrance and Exit Criteria
• Address unforeseen events:
  • Establish a framework for corrective actions.
  • Ensure that all reviews (e.g., design reviews, progress audits) are explicitly listed in the SOW and contract documents.
• For reviews not initially covered in the SOW, amendments may be necessary to incorporate them post-contract award.

Additional Considerations

1. Consult Center guidance to supplement milestone selection, review approaches, and scheduling strategies.
2. Refer to Topic 7.09 to define entrance and exit criteria while developing milestone checklists. See 7.09 - Entrance and Exit Criteria
3. Collaborate with the technical authority and stakeholders during planning to ensure alignment with project needs.

Conclusion

Milestone reviews are critical to monitoring progress, evaluating deliverables, managing risks, and maintaining alignment with project goals and schedules. By carefully selecting, documenting, and managing milestones, project managers can balance oversight with efficiency, ensuring mission success while fostering a strong foundation for project accountability and quality.

3.6 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

3.7 Center Process Asset Libraries

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only). 

4. Small Projects

Small projects often differ from larger, more complex efforts in terms of scope, resources, and timeline. However, maintaining structured review plans and milestones is equally critical to ensuring progress, achieving technical goals, and meeting stakeholder expectations. The following improved guidance refines this process by providing tailored advice for small projects to enable efficient oversight without introducing unnecessary complexity.

1. Importance of Milestones and Project Reviews

Small projects are required to adhere to NASA’s established guidelines from NPR 7120.5, NPR 7120.7, NPR 7120.8, and NPR 7123.1. These documents define milestone reviews and emphasize the integration of software components into specific project reviews. While smaller projects may not require the extensive documentation or frequency of reviews seen in larger projects, aligning software progress with key milestones remains essential for meeting mission objectives and ensuring technical direction.

Key Advantages for Small Projects:

• Visibility into Progress: Milestones provide clear checkpoints to assess whether the development effort is on track.
• Alignment with Technical Goals: Regular reviews ensure the project is progressing according to the technical and operational expectations defined during formulation.
• Early Risk Detection: Review milestones serve as opportunities to identify issues (technical, resource-related, or otherwise) before they impact project outcomes.
• Focus on Mission Criticality: Small projects typically have limited resources; milestones help prioritize essential tasks and streamline efforts on meeting critical deliverables.

2. Establishing Review Milestones for Small Projects

Small projects should determine a review process tailored to their scope and complexity. This process should:

1. Reflect Project Requirements: Ensure milestones and reviews align with the specific technical and customer requirements of the project.
2. Promote Simplicity and Efficiency: Streamline review activities to focus on the greatest insights into progress without overburdening the team.
3. Integrate Software Deliverables: Explicitly include software components in milestone planning to ensure compliance, traceability, and proper oversight.

Guidelines for Tailoring Review Milestones:

• Simplify the Review Process: Aim for fewer, high-impact reviews centered on critical project goals rather than exhaustive evaluations.
• Combine Phases Where Appropriate: If certain project phases (design, coding, testing) are straightforward or well-defined, their reviews can be conducted together.
• Ensure Adequate Content: Include only the most essential technical metrics, risks, and progress indicators in review discussions.
• Balance Resources and Insights: The review process must be robust enough to provide meaningful insights while remaining practical for the size of the project.

3. Suggested Review Milestones for Small Projects

Typical milestone reviews for small projects include the following:

Formulation Phase:

1. Project Kickoff/Initial Requirements Review:
   • Discuss customer needs, project scope, and establish technical goals.
   • Define entrance and exit criteria for subsequent milestones.

Development Phase:

1. Software Requirements Review (SRR):

   • Validate that system-level requirements are fully understood and adequately translated into software specifications.
   • Identify gaps or risks in requirement traceability.

2. Preliminary Design Review (PDR):

   • Confirm technical feasibility and alignment with project goals.
   • Review initial design elements, tools, and technologies for software implementation.

3. Critical Design Review (CDR):

   • Evaluate the final software design and readiness for coding.
   • Ensure alignment with operational requirements, including safety and performance benchmarks.

4. Test Readiness Review (TRR):

   • Assess testing plans, test cases, and resources for the verification and validation of software deliverables.

Delivery Phase:

1. System Integration Review (SIR):

   • Evaluate how the software integrates with hardware systems, legacy systems, or other mission elements.
   • Identify any interoperability risks.

2. Acceptance/Deployment Review:

   • Confirm software readiness for delivery and operational use.
   • Validate compliance with milestones, regulatory requirements, and customer expectations.

4. Practical Steps for Structuring Review Processes

To develop a tailored review process that meets project requirements, consider the following steps:

1. Identify Critical Milestones

• Narrow down review points to track essential technical and operational aspects of the project.
• Focus on deliverables that affect mission success, such as requirements validation, test readiness, and software integration.

2. Prioritize Content Over Documentation

• Optimize review agendas by focusing on metrics, risks, and goals pertinent to the size of the project.
• Simplify reporting by using templates or condensed summaries of milestone progress.

3. Include Stakeholders and Customers

• Engage relevant stakeholders in the review process to ensure alignment and buy-in on technical direction.
• Keep communication clear and precise for non-technical stakeholders to ensure shared understanding of progress and concerns.

4. Tailor Oversight for Small Teams

• Reduce administrative burden by incorporating agile feedback loops where appropriate.
• Avoid requiring extensive documentation for reviews unless mandated by external contracts or regulatory compliance.

5. Use NASA Resources

• Small projects should leverage existing NASA resources, such as Center Process Asset Libraries (PALs) or frameworks from NPR 7120.5 and NPR 7123.1, to simplify milestone planning and execution.

5. Special Considerations for Acquired Software Development

When acquiring software, milestone reviews must be incorporated into contracts and development agreements to enforce contractor compliance and performance. For small projects, the following contractual elements are essential:

• Defined Surveillance Activities: Include monitoring activities, milestone reviews, technical audits, and decision points in the contract to ensure progress visibility.
• Review and Approval Periods: Specify deadlines for deliverable submissions and required corrections to resolve findings.
• Entrance and Exit Criteria: Ensure formal checklists for milestone approvals are included, enabling clarity for both the contractor and project team.

Recommended Contractual Elements:

• Technical Reviews (as defined in NPR 7120.5, 7120.7, or 7123.1).
• Review completeness checklists to verify milestone outputs.
• Progress reporting templates tied to milestones for efficient oversight.

6. Reference Guidance for Small Projects

Small projects can refer to the following documents for additional insights and requirements:

1. NPR 7120 Family: Defines milestone review standards and approaches for different project types.
2. Topic 7.09: Describes review entrance and exit criteria, inputs, reviewed materials, and outputs.
3. Acquisition Guidance (7.03): Provides strategies for incorporating milestone reviews into contracts.
4. Issue Management Systems: Use automated tools to track and resolve action items generated during milestone reviews.

Conclusion

Small projects must still maintain structured review processes and milestones to ensure alignment with technical goals and mission success. Tailored, efficient planning allows for adequate oversight without excessive administrative overhead, providing the greatest insights into progress, risks, and technical direction. By leveraging simplified review processes, prioritizing critical content, and engaging stakeholders effectively, small projects can achieve strong outcomes while remaining resource-efficient.

5. Resources

5.1 References

5.2 Tools

6. Lessons Learned

6.1 NASA Lessons Learned

A documented lesson from the NASA Lessons Learned database notes the following:

• Acquisition and Oversight of Contracted Software Development (1999). Lesson Number 0921⁵²⁸: Tailorable acquisition management and oversight processes for NASA contracted software development are essential to ensure that customers receive a quality product. A documented lesson from the NASA Lessons Learned database includes a cause of the loss of a mission "the lack of a controlled and effective process for acquisition of contractor-developed, mission-critical software." In this particular case, the quality of the contractor's product was not monitored as it would have been if the proper milestones for reviewing and auditing contractor progress were in place.

6.2 Other Lessons Learned

The Goddard Space Flight Center (GSFC) Lessons Learned online repository ⁶⁹⁵ contains the following lessons learned related to software requirements identification, development, documentation, approval, and maintenance based on analysis of customer and other stakeholder requirements and the operational concepts. Select the titled link below to access the specific Lessons Learned:

• All stakeholders should be aware of the project plan and schedule. Lesson Number 63: The recommendation states: "All stakeholders should be aware of the project plan and schedule."
• Assess status independently based on artifacts, not on reported status. Lesson Number 141: The recommendation states: "Assess status independently based on artifacts, not on reported status."

7. Software Assurance

7.1 Tasking for Software Assurance

7.2 Software Assurance Products

•  Software Assurance Status Reports
• SA audit schedule is consistent with the project schedule.
• List of any issues//risks identified with schedule or developer progress.

  
  

  Objective Evidence

  • Milestone review plans.
  • Definition for software products life cycle entrance and exit criteria.
  • Evidence of SA participation in milestone reviews.
  Definition of objective evidence

  Objective evidence is an unbiased, documented fact showing that an activity was confirmed or performed by the software assurance/safety person(s). The evidence for confirmation of the activity can take any number of different forms, depending on the activity in the task. Examples are:

  • Observations, findings, issues, risks found by the SA/safety person and may be expressed in an audit or checklist record, email, memo or entry into a tracking system (e.g. Risk Log).
  • Meeting minutes with attendance lists or SA meeting notes or assessments of the activities and recorded in the project repository.
  • Status report, email or memo containing statements that confirmation has been performed with date (a checklist of confirmations could be used to record when each confirmation has been done!).
  • Signatures on SA reviewed or witnessed products or activities, or
  • Status report, email or memo containing a short summary of information gained by performing the activity. Some examples of using a “short summary” as objective evidence of a confirmation are:
    • To confirm that: “IV&V Program Execution exists”, the summary might be: IV&V Plan is in draft state. It is expected to be complete by (some date).
    • To confirm that: “Traceability between software requirements and hazards with SW contributions exists”, the summary might be x% of the hazards with software contributions are traced to the requirements.
  • The specific products listed in the Introduction of 8.16 are also objective evidence as well as the examples listed above.

SA Products for Milestone Reviews

See also Topic 8.05 - SW Failure Modes and Effects Analysis, 8.07 - Software Fault Tree Analysis, 8.12 - Basics of Software Auditing

7.3 Metrics

• # of Non-Conformances from reviews (Open vs. Closed; # of days Open)

7.4 Guidance

List of tasks for the software development required for the project’s software developers. - look at the software development schedule milestones (see 7.08 - Maturity of Life Cycle Products at Milestone Reviews, software products required, and software processes used. Related to the SA tasks on SWE-036 - Software Process Determination. 

7.5 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook: