Renew your license to continue

Your evaluation license of Visibility for Confluence expired. Please use the Buy button to purchase a new license.

Renew your license to continue

Your evaluation license has expired. Contact your administrator to renew your Scaffolding Forms & Templates license.

This "Review" topic is provided to demonstrate a suggested change to the page.

The suggested change is implemented in the "PDR" tab to show the following changes.

Review Copy of 7.09 - Entrance and Exit Criteria

Web Resources

View this section on the website

See edit history of this section

Post feedback on this section

Section Labels:

Unknown macro: {page-info}

Introduction
SDR

Entrance and Exit Criteria

Background

This guidance provides the recommended life cycle review entrance and exit criteria for software projects and should be tailored for the project class.

This topic describes the recommended best practices for entrance and success criteria for the software life cycle and technical reviews required by NASA. The guidelines are regardless of whether the review is accomplished in a one-step or two-step process. The entrance and items reviewed criteria do not provide a complete list of all products and their required maturity levels. Additional programmatic products may also be required by the appropriate governing NPRs for the project/program.

Tailoring and customizing are expected for projects and programs. The entrance and success criteria and products required for each review will be tailored and customized appropriately for the particular program or project being reviewed and the classification of the software. The decisions made to tailor and customize life-cycle review criteria should be justified to both the Engineering and SMA TA.

The recommended criteria in the following tables are focused on demonstrating acceptable software technical maturity, adequacy of software technical planning and credibility of budget, software schedule, risks (as applicable), and software readiness to proceed to the next phase. Customized or tailored criteria developed by programs or projects for life-cycle reviews should also be focused on assessing these factors. The software entrance and exit criteria guidance is a collection of material from the following core documents: NPR 7150,2 ⁰⁸³ requirements, NASA-STD-8739.8 ²⁷⁸ requirements, and NPR 7123.1, Appendix G ⁰⁴¹.

1.1 References

Click here to view master references table.

Renew your license to continue

Your evaluation license has expired. Contact your administrator to renew your Reporting for Confluence license.

Renew your license to continue

Your evaluation license of Visibility for Confluence expired. Please use the Buy button to purchase a new license.

1.2 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

Related Links
7.08 - Maturity of Life Cycle Products at Milestone Reviews 8.02 - Software Reliability 8.09 - Software Safety Analysis

1.3 Center Process Asset Libraries

SPAN - Software Processes Across NASA
SPAN contains links to Center managed Process Asset Libraries. Consult these Process Asset Libraries (PALs) for Center-specific guidance including processes, forms, checklists, training, and templates related to Software Development. See SPAN in the Software Engineering Community of NEN. Available to NASA only. https://nen.nasa.gov/web/software/wiki ¹⁹⁷

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only).

SPAN Links
Project Planning

Renew your license to continue

Your evaluation license of Visibility for Confluence expired. Please use the Buy button to purchase a new license.

System Definition Review (SDR)

The MDR (or SDR) examines the proposed requirements, the mission/system architecture, and the flow down to all functional elements of the system. (NPR 7120.5 ⁰⁸²)

MDR is equivalent to SDR for robotic projects.

Entrance Criteria
Software requirements
Bi-directional traceability
Completed Software Plans
Completed software classification and safety criticality assessments are available
Identification of software safety-critical components by having a preliminary software hazard analysis completed
NASA NPR 7150.2 requirements mapping matrix is complete
NASA-STD-8739.8 requirements mapping matrix is complete
Software data dictionary
Completed Software Safety Assessment
Completed software assurance software requirements analysis
Completed the IV&V software requirements analysis
Completed peer review(s) of the software requirements
Completed peer review(s) of the software plans
Preliminary Human Rating Plan, if applicable
Preliminary Maintenance Plan
Software configuration management (CM) plan
Completed IV&V Project Execution Plan
Confirm RFAs and RIDs from MCR have been satisfactorily resolved
Identify the software assurance, software safety, and IV&V personnel for the project and milestone review.
The software assurance point of contact for the project has been identified.

Items Reviewed
Requirements mapping table for the software assurance and software safety standard requirements
Software requirements
Interface requirements documents, including Software interfaces
Hazard Analysis and software controls and mitigations (Functional Hazard Analysis / Hazard Reports / Hazard Analysis Tracking Index)
Software and avionics architectures
Software data dictionary
Software assurance requirement analysis results
Software classifications
Software architecture
Software Management Plan(s)
Software process audit results
Software verification and validation (V&V) planning
Bidirectional traceability matrix.
Technical resource utilization estimates and margins
Software configuration management (CM) plan
Software Assurance (SA) and Software Safety Plan(s), including SA Product Acceptance Criteria and Conditions.
Software Development Plan
Software peer review results
Software Assurance schedule
Software coding guidelines
Software risks
Software safety analysis results
Cost estimate for the project’s Software Assurance support
IV&V Project Execution Plan (if required)
IV&V Project risk assessment (if required)
System architecture, including avionics architecture
Top technical, cost, schedule, and safety risks, risk mitigation plans, and associated resources
Concept Documentation

Exit / Success Criteria
The project’s software plans, schedules, resources, and requirements are sufficiently mature to begin Phase B. Click here to expand... Adequate planning exists for developing, inserting, or deploying any enabling new software technology or avionics technology needed for software development. The software schedule satisfies the following conditions: Coordinates with the overall project schedule, Documents the interactions of milestones and deliverables between software, hardware, operations, and the rest of the system, Reflects the critical dependencies for software development activities and identifies and accounts for dependencies with other projects and cross-program dependencies. The proposed avionics/software architecture is credible and responsive to program requirements and constraints, including supporting the single-point failure/Fault Tolerance requirements. The planned use of software static code analysis is acceptable.
Quality software requirements have been established. Click here to expand... Software requirements include requirements for COTS, GOTS, MOTS, OSS, reused software components, interface requirements, fault management software requirements, and software-related safety constraints, controls, and mitigations. The software requirements are testable. The software requirements address the Software Fault Tree Analysis (FTA) or Software Failure Modes and Effects Analysis results. TBD and TBR items are identified with acceptable plans and schedules for their disposition, and the software volatility metric is less than 30%. The % of TBD/TBC/TBRs in the software requirements document does not exceed 20% High software requirements quality risk score (software requirements quality score of 3 or higher). Less than 3 is a risk, and fewer than 20% of the requirements have a high-risk or very high-risk score. A sufficient number of detailed software requirements exist. Software cybersecurity requirements are defined. Operational scenarios are defined enough to support the definition and allocation of software requirements.
Software Bi-directional traceability is complete. Click here to expand... Bi-directional traceability is complete with the system, hardware, and ICD requirements. Transparent allocation of system requirements to software and software architectural elements.
System Hazard Analyses address or include all known software hazards, Click here to expand... The software hazards are clearly defined. Traceability between software requirements and hazards is complete. Hazard controls are defined. Hazard verifications are defined. All software requirements that trace to a hazard analysis are to be verified by test. Software hazards address the software Failure Modes and Effects Analysis (FEMAs) and software Fault Tree results.
The software NPR 7150.2 and NASA-STD-8739.8 requirements tailoring have been completed and approved by the required technical authorities and the project. Click here to expand... The correct software classifications have been defined.
Certifiable software development practices by the organizations developing the critical software components exist.
Software metrics to track the software quality and maturity have been selected. Click here to expand... The software technical metric margins are defined and are acceptable. Adequate technical and programmatic margins (e.g., data throughput, memory, CPU utilization) and resources exist to complete the software development within budget, schedule, and known risks
Certifiable software development practices by the organizations developing the critical software components exist and are being followed. Click here to expand... Evidence exists with Software process audit results. Evidence that the software development organization followed the required processes for this point in the software lifecycle.
All known software risks are identified and technically assessed. Click here to expand... NASA has access to the software products in electronic format, including software development and management metrics. The planned use of software static code analysis is acceptable. Software risks, issues, and findings are documented. The project has addressed all software assurance and Severity 1 or 2 findings. Verify that the project does not have a risk associated with developing software at the same time as the hardware is being designed, the risk of misunderstanding the software-hardware interfaces, and not having the hardware available to use in the software development. Software supply chain risks. Sufficient software workforce or software skillsets exist.
IV&V concurs that the project’s software plans, schedules, resources, and requirements are sufficiently mature to begin Phase B. (If IV&V is required)

Entrance Criteria	Items Reviewed	Exit / Success Criteria
Software requirements	Requirements mapping table for the software assurance and software safety standard requirements	The project’s software plans, schedules, resources, and requirements are sufficiently mature to begin Phase B. Click here to expand... Adequate planning exists for developing, inserting, or deploying any enabling new software technology or avionics technology needed for software development. The software schedule satisfies the following conditions: Coordinates with the overall project schedule, Documents the interactions of milestones and deliverables between software, hardware, operations, and the rest of the system, Reflects the critical dependencies for software development activities and identifies and accounts for dependencies with other projects and cross-program dependencies. The proposed avionics/software architecture is credible and responsive to program requirements and constraints, including supporting the single-point failure/Fault Tolerance requirements. The planned use of software static code analysis is acceptable.
Bi-directional traceability	Software requirements	Quality software requirements have been established. Click here to expand... Software requirements include requirements for COTS, GOTS, MOTS, OSS, reused software components, interface requirements, fault management software requirements, and software-related safety constraints, controls, and mitigations. The software requirements are testable. The software requirements address the Software Fault Tree Analysis (FTA) or Software Failure Modes and Effects Analysis results. TBD and TBR items are identified with acceptable plans and schedules for their disposition, and the software volatility metric is less than 30%. The % of TBD/TBC/TBRs in the software requirements document does not exceed 20% High software requirements quality risk score (software requirements quality score of 3 or higher). Less than 3 is a risk, and fewer than 20% of the requirements have a high-risk or very high-risk score. A sufficient number of detailed software requirements exist. Software cybersecurity requirements are defined. Operational scenarios are defined enough to support the definition and allocation of software requirements.
Completed Software Plans	Interface requirements documents, including Software interfaces	Software Bi-directional traceability is complete. Click here to expand... Bi-directional traceability is complete with the system, hardware, and ICD requirements. Transparent allocation of system requirements to software and software architectural elements.
Completed software classification and safety criticality assessments are available	Hazard Analysis and software controls and mitigations (Functional Hazard Analysis / Hazard Reports / Hazard Analysis Tracking Index)	System Hazard Analyses address or include all known software hazards, Click here to expand... The software hazards are clearly defined. Traceability between software requirements and hazards is complete. Hazard controls are defined. Hazard verifications are defined. All software requirements that trace to a hazard analysis are to be verified by test. Software hazards address the software Failure Modes and Effects Analysis (FEMAs) and software Fault Tree results.
Identification of software safety-critical components by having a preliminary software hazard analysis completed	Software and avionics architectures	The software NPR 7150.2 and NASA-STD-8739.8 requirements tailoring have been completed and approved by the required technical authorities and the project. Click here to expand... The correct software classifications have been defined.
NASA NPR 7150.2 requirements mapping matrix is complete	Software data dictionary	Certifiable software development practices by the organizations developing the critical software components exist.
NASA-STD-8739.8 requirements mapping matrix is complete	Software assurance requirement analysis results	Software metrics to track the software quality and maturity have been selected. Click here to expand... The software technical metric margins are defined and are acceptable. Adequate technical and programmatic margins (e.g., data throughput, memory, CPU utilization) and resources exist to complete the software development within budget, schedule, and known risks
Software data dictionary	Software classifications	Certifiable software development practices by the organizations developing the critical software components exist and are being followed. Click here to expand... Evidence exists with Software process audit results. Evidence that the software development organization followed the required processes for this point in the software lifecycle.
Completed Software Safety Assessment	Software architecture	All known software risks are identified and technically assessed. Click here to expand... NASA has access to the software products in electronic format, including software development and management metrics. The planned use of software static code analysis is acceptable. Software risks, issues, and findings are documented. The project has addressed all software assurance and Severity 1 or 2 findings. Verify that the project does not have a risk associated with developing software at the same time as the hardware is being designed, the risk of misunderstanding the software-hardware interfaces, and not having the hardware available to use in the software development. Software supply chain risks. Sufficient software workforce or software skillsets exist.
Completed software assurance software requirements analysis	Software Management Plan(s)	IV&V concurs that the project’s software plans, schedules, resources, and requirements are sufficiently mature to begin Phase B. (If IV&V is required)
Completed the IV&V software requirements analysis	Software process audit results
Completed peer review(s) of the software requirements	Software verification and validation (V&V) planning
Completed peer review(s) of the software plans	Bidirectional traceability matrix.
Preliminary Human Rating Plan, if applicable	Technical resource utilization estimates and margins
Preliminary Maintenance Plan	Software configuration management (CM) plan
Software configuration management (CM) plan	Software Assurance (SA) and Software Safety Plan(s), including SA Product Acceptance Criteria and Conditions.
Completed IV&V Project Execution Plan	Software Development Plan
Confirm RFAs and RIDs from MCR have been satisfactorily resolved	Software peer review results
Identify the software assurance, software safety, and IV&V personnel for the project and milestone review.	Software Assurance schedule
The software assurance point of contact for the project has been identified.	Software coding guidelines
	Software risks
	Software safety analysis results
	Cost estimate for the project’s Software Assurance support
	IV&V Project Execution Plan (if required)
	IV&V Project risk assessment (if required)
	System architecture, including avionics architecture
	Top technical, cost, schedule, and safety risks, risk mitigation plans, and associated resources
	Concept Documentation

Additional Supporting Material

Entrance Criteria

Successful completion of the previous review (typically SRR) and responses made to all Requests for Actions (RFAs) and Review Item Discrepancies (RIDs)
Final agenda, success criteria, and charge to the board have been agreed to by the technical team, project manager, and review chair
Technical products for this review were made available to participants prior to SDR
Preferred software solution defined including major tradeoffs and options
Baseline documentation updated, as required
Preliminary functional baseline (with supporting trade-off analyses and data) available
Preliminary system software functional requirements available
As applicable, the risk management plan updated (could be part of SDP/SMP)
- Updated software risk assessment and mitigations (including Probabilistic Risk Assessment (PRA), as applicable)
As applicable, SDP/SMP updated
- Updated technology development, maturity, and assessment plan
- Updated cost and schedule data
- Work Breakdown Structure
Preliminary software safety analysis available for review
Project software data dictionary available
Preliminary project software assurance plan available
As applicable, an updated project software maintenance plan is available
Software inputs/contributions completed for:
- Flow down of system requirements to all software functional elements of the system
- Requirements process
- Technical approach

Software Assurance:

Confirm that all RIDs/RFAs from SwRR have been resolved
Coordinate with the project on any changes in software classification
Review or participate in system preliminary safety analysis
Confirm that:
- The preferred software solution is defined as including major tradeoffs and options
- The functional baseline is available; system software functional requirements exist and have been flowed down into software functional elements of the system.
- The requirements process is in place
- SDP/SMP has been updated (as applicable), including risk management, cost, schedule, work break-down structure, technical management, plans for monitoring performance
- Confirm other baselined documentation has been updated as needed; check for alignment of documentation
- Complete contributions to system safety and mission assurance plans
- Complete software assurance plan
- Confirm those entrance criteria are met
- Review materials submitted for the review

Items Reviewed

System architecture, including software
Preferred software solution with tradeoffs and options
Preliminary functional baseline
Preliminary system software functional requirements
The risk management plan, as applicable
SDP/SMP, as applicable
Preliminary software verification and validation (V&V) planning, as applicable
Software requirements documents
Interface requirements documents, including SW
Technical resource utilization estimates and margins
Software safety analysis
Software data dictionary
Software configuration management (CM) plan
Software quality assurance (QA) plan

Software Assurance:

Review materials for review; attend the review
Summarize or present any SA analysis or audit results from this phase
Submits any RIDs/RFAs for identified issues or risks

Success Criteria

The review panel agrees that:
- Software requirements, including mission success criteria and any sponsor-imposed constraints, are defined and form the basis for the proposed conceptual design
- System-level requirements are flowed down to the software
- All software technical requirements are allocated and flow down to subsystems is adequate; they are verifiable and traceable to their corresponding system level requirement; requirements, design approaches, and conceptual design will fulfill the mission needs consistent with the available resources (cost, schedule, throughput, and sizing)
- Technical plans have been updated, as necessary, including a risk management plan, SDP/SMP, V&V planning, software maintenance plan, QA plan, CM plan
- Tradeoffs are completed, and those planned for Phase B adequately address the option space
- Adequate planning exists for the development of any enabling new technology
- Significant development, mission, and safety risks are identified and technically assessed, and a process and resources exist to manage the risks
- The operations concept is consistent with the proposed design concept(s) and in alignment with the mission requirements
- The requisite level of detail and resources are available to support the acquisition and development plan within existing constraints
- All of these software subsystem requirements are traceable to either mission objectives, the concept of operations, or interface requirements
- Monitoring processes/practices are in place to create a software subsystem within planned technical, schedule, cost, effort, and quality capabilities
Preliminary verification approaches are agreed upon

Software Assurance:

Confirm all risks and issues are documented.
RIDs/RFAs for the SA Plan implemented
Agree with the review panel that the items above have been adequately addressed
Agree with resolutions or action plans for RFAs/RIDs; track them to closure

Content

Space Tools

Entrance and Exit Criteria

Background

1.1 References

1.2 Additional Guidance

1.3 Center Process Asset Libraries

System Definition Review (SDR)

Additional Supporting Material

Software Assurance:

Software Assurance:

Software Assurance: