Page History
...
Table 1. Software Assurance and Software Safety Requirements Mapping Matrix
| NPR 7150.2 Section | SWE # | NPR 7150.2 Requirement | Software Assurance and Software Safety Tasks |
|---|---|---|---|
| 3 | Software Management Requirements | ||
| 3.1 | Software Life-Cycle Planning | ||
| 3.1.2 | 033 | The project manager shall assess options for software acquisition versus development. | 1. Confirm that the options for software acquisition versus development have been evaluated. 2. Confirm the flow down of applicable software engineering, software assurance, and software safety requirements on all acquisition activities. (NPR 7150.2 and NASA-STD-8739.8). 3. Assess any risks with acquisition versus development decision(s). |
| 3.1.3 | 013 | The project manager shall develop, maintain, and execute software plans that cover the entire software life-cycle and, as a minimum, address the requirements of this directive with approved tailoring. | 1. Confirm that all plans are in place, and have expected content for the life-cycle events, with proper tailoring for the classification of the software. 2. Develop a Software Assurance Plan following the content defined in NASA-HDBK-2203 for a software assurance plan, including software safety. |
| 3.1.4 | 024 | The project manager shall track the actual results and performance of software activities against the software plans. a. Corrective actions are taken, recorded, and managed to closure. b. Including changes to commitments (e.g., software plans) that have been agreed to by the affected groups and individuals. | 1. Assess plans for compliance with NPR 7150.2 requirements, NASA-STD-8739.8, including changes to commitments. 2. Confirm that closure of corrective actions associated with the performance of software activities against the software plans, including closure rationale. |
| 3.1.5 | 034 | The project manager shall define and document the acceptance criteria for the software. | 1. Confirm software acceptance criteria are defined and assess the criteria based on guidance in the NASA Software Engineering Handbook, NASA-HDBK-2203. |
...
| 3.1.6 | 036 | The project manager shall establish and maintain the software processes, software documentation plans, list of developed electronic products, deliverables, and list of tasks for the software development that are required for the project’s software developers, as well as the action required (e.g., approval, review) of the Government upon receipt of each of the deliverables. | 1. Confirm the following are approved, implemented, and updated per requirements: a. Software processes, including software assurance, software safety, and IV&V processes. 2. Confirm that any required government actions upon receipt of deliverables (e.g., approvals, reviews) are established and maintained. |
| 3.1.7 | 037 | The project manager shall define and document the milestones at which the software developer(s) progress will be reviewed and audited. | 1. Confirm that milestones for reviewing and auditing software developer progress are defined and documented. 2. Participate in project milestones reviews. |
| 3.1.8 | 039 | The project manager shall require the software developer(s) to periodically report status and provide insight into software development and test activities; at a minimum, the software developer(s) will be required to allow the project manager and software assurance personnel to: a. Monitor product integration. b. Review the verification activities to ensure adequacy. c. Review trade studies and source data. d. Audit the software development processes and practices. e. Participate in software reviews and technical interchange meetings. | 1. Confirm that software developer(s) are periodically reporting status and providing insight to the project manager. 2. Monitor product integration. 3. Analyze the verification activities to ensure adequacy. 4. Assess trade studies, source data, software reviews, and technical interchange meetings. 5. Perform audits on software development processes and practices at least once every two years. 6. Develop and provide status reports. 7. Develop and maintain a list of all software assurance review discrepancies, risks, issues, findings, and concerns |
...
| . |
...
8. Confirm that the project manager provides responses to software assurance and software safety submitted issues, findings, and risks and that the project manager tracks software assurance and software safety issues, findings, and risks to closure. | |||
| 3.1.9 | 040 | The project manager shall require the software developer(s) to provide NASA with software products, traceability, software change tracking information and nonconformances, in electronic format, including software development and management metrics. | 1. Confirm that software artifacts are available in electronic format to NASA. |
| 3.1.10 | 042 | The project manager shall require the software developer(s) to provide NASA with electronic access to the source code developed for the project in a modifiable format. | 1. Confirm that software developers are providing NASA with electronic access to the source code generated for the project in a modifiable form. |
| 3.1.11 | 139 | The project manager shall comply with the requirements in this NPR that are marked with an ”X” in Appendix C consistent with their software classification. | 1. Assess that the software requirements, products, procedures, and processes of the project are compliant with the NPR 7150.2 requirements per the software classification and safety criticality for software. |
| 3.1.12 | 121 | Where approved, the project manager shall document and reflect the tailored requirement in the plans or procedures controlling the development, acquisition, and deployment of the affected software. | 1. Confirm that any requirement tailoring in the Requirements Mapping Matrix has the required approvals. 2. Develop a tailoring matrix of software assurance and software safety requirements. |
| 3.1.13 | 125 | Each project manager with software components shall maintain a requirements mapping matrix or multiple requirements mapping matrices against requirements in this NPR, including those delegated to other parties or accomplished by contract vehicles or Space Act Agreements. | 1. Confirm that the project is maintaining a requirement mapping matrix (matrices) for all requirements in NPR 7150.2. 2. Maintain the requirement mapping matrix (matrices) for requirements in NASA-STD-8739. |
...
| 8. | |||
| 3.1.14 | 027 | The project manager shall satisfy the following conditions when a Commercial-Off-The-Shelf (COTS), Government Off-The-Shelf (GOTS), Modified Off-The-Shelf (MOTS), Open Source Software (OSS), or reused software component is acquired or used: a. The requirements to be met by the software component are identified. | 1. Confirm that the conditions listed in "a" through "f" are complete for any COTS, GOTS, MOTS, OSS, or reused software that is acquired or used. |
| 3.2 | Software Cost Estimation |
...
| 3.2.1 | 015 | To better estimate the cost of development, the project manager shall establish, document, and maintain: a. Two cost estimate models and associated cost parameters for all Class A and B software projects that have an estimated project cost of $2 million or more. b. One software cost estimate model and associated cost parameter(s) for all Class A and Class B software projects that have an estimated project cost of less than $2 million. c. One software cost estimate model and associated cost parameter(s) for all C and D software projects. d. One software cost estimate model and associated cost parameter(s) for all Class F software projects. | 1. Confirm that the required number of software cost estimates are complete and include software assurance cost estimate(s) for the project, including a cost estimate associated with handling safety-critical software and safety-critical data |
...
| . |
...
| 3.2.2 | 151 | The project manager’s software cost estimate(s) shall satisfy the following conditions: a. Covers the entire software life-cycle. b. Is based on selected project attributes (e.g., assessment of the size, functionality, complexity, criticality, reuse code, modified code, and risk of the software processes and products). c. Is based on the cost implications of the technology to be used and the required maturation of that technology. d. Incorporates risk and uncertainty, including cybersecurity. e. Includes the cost of the required software assurance support. f. Includes other direct costs. | 1. Assess the project's software cost estimate(s) to determine if the stated criteria listed in "a" through "f" are satisfied. |
| 3.2.3 | 174 | The project manager shall submit software planning parameters, including size and effort estimates, milestones, and characteristics, to the Center measurement repository at the conclusion of major milestones. | 1. Confirm that all the software planning parameters, including size and effort estimates, milestones, and characteristics, are submitted to a Center repository. 2. Confirm that all software assurance and software safety software estimates and planning parameters are submitted to an organizational repository. |
| 3.3 | Software Schedules |
...
| 3.3.1 | 016 | The project manager shall document and maintain a software schedule that satisfies the following conditions: a. Coordinates with the overall project schedule. b. Documents the interactions of milestones and deliverables between software, hardware, operations, and the rest of the system. c. Reflects the critical dependencies for software development activities. d. Identifies and accounts for dependencies with other projects and cross-program dependencies. | 1. Assess that the software schedule satisfies the conditions in the requirement. 2. Develop a software assurance schedule, including software assurance products, audits, reporting, and reviews. |
| 3.3.2 | 018 | The project manager shall regularly hold reviews of software schedule activities, metrics, status, and results with the project stakeholders and track issues to resolution. | 1. Confirm the generation and distribution of periodic reports on software schedule activities, metrics, and status, including reports of software assurance and software safety schedule activities, metrics, and status. 2. Confirm closure of any project software schedule issues. |
| 3.3.3 | 046 | The project manager shall require the software developer(s) to provide a software schedule for the project's review, and schedule updates as requested. | 1. Confirm the project's schedules are updated. |
| 3.4 | Software Training | ||
| 3.4.1 | 017 | The project manager shall plan, track, and ensure project specific software training for project personnel. | 1. Confirm that any project-specific software training has been planned, tracked, and completed for project personnel, including software assurance and software safety personnel. |
| 3.5 | Software Classification Assessments |
...
| 3.5.1 | 020 | The project manager shall classify each system and subsystem containing software in accordance with the highest applicable software classification definitions for Classes A, B, C, D, E, and F software in Appendix D. | 1. Perform a software classification or concur with the engineering software classification of software per the descriptions in NPR 7150.2. |
| 3.5.2 | 176 | The project manager shall maintain records of each software classification determination, each software Requirements Mapping Matrix, and the results of each software independent classification assessment for the life of the project. | 1. Confirm that records of the software Requirements Mapping Matrix and each software classification are maintained and updated for the life of the project. |
| 3.6 | Software Assurance and Software Independent Verification & Validation | ||
| 3.6.1 | 022 | The project manager shall plan and implement software assurance per NASA-STD-8739.8. | 1. Perform according to the software assurance plan and the software assurance and software safety standard requirements in NASA-STD-8739.8. |
| 3.6.2 | 141 | For projects reaching Key Decision Point A, the program manager shall ensure that software IV&V is performed on the following categories of projects: a. Category 1 projects as defined in NPR 7120.5. b. Category 2 projects as defined in NPR 7120.5 that have Class A or Class B payload risk classification per NPR 8705.4. c. Projects selected explicitly by the NASA Chief of the Office of Safety and Mission Assurance to have software IV&V. | 1. Confirm that IV&V requirements (section 4.4) are complete on projects that are required to have IV&V. |
| 3.6.3 | 131 | If software IV&V is performed on a project, the project manager shall ensure an IPEP is developed, negotiated, approved, maintained, and executed. | 1. Confirm that the IV&V Program Execution Plan (IPEP) exists. |
...
| 3.6.4 | 178 | If software IV&V is performed on a project, the project manager shall ensure that IV&V is provided access to development artifacts, products, source code, and data required to perform the IV&V analysis efficiently and effectively. | 1. Confirm that IV&V has access to the software development artifacts, products, source code, and data required to perform the IV&V analysis efficiently and effectively. |
| 3.6.5 | 179 | If software IV&V is performed on a project, the project manager shall provide responses to IV&V submitted issues and risks, and track these issues and risks to closure. | 1. Confirm that the project manager provides responses to IV&V submitted issues, findings, and risks and that the project manager tracks IV&V issues, findings, and risks to closure. |
| 3.7 | Safety-critical Software | ||
| 3.7.1 | 205 | The project manager, in conjunction with the SMA organization, shall determine if each software component is considered to be safety-critical per the criteria defined in NASA-STD-8739.8. | 1. Confirm that the hazard reports or safety data packages contain all known software contributions or events where software; either by its action, inaction or incorrect action, lead to a hazard. 2. Assess that the hazard reports identify the software components associated with the system hazards per the criteria defined in NASA-STD- 8739.8, Appendix A. 3. Assess that hazard analyses (including hazard reports) identify the software components associated with the system hazards per the criteria defined in NASA-STD- 8739.8, Appendix A. 4. Confirm that the traceability between software requirements and hazards with software contributions exist. 5. Develop and maintain a software safety analysis throughout the software development life-cycle |
...
| . |
...
| 3.7.2 | 023 | If a project has safety-critical software, the project manager shall implement the safety-critical software requirements contained in NASA-STD-8739.8. | 1. Confirm that the identified safety-critical software components and data have implemented the safety-critical software assurance requirements listed in this standard. |
| 3.7.3 | 134 | If a project has safety-critical software or mission-critical software, the project manager shall implement the following items in the software: a. The software is initialized, at first start and restarts, to a known safe state. b. The software safely transitions between all predefined known states. c. Termination performed by software of functions is performed to a known safe state. d. Operator overrides of software functions require at least two independent actions by an operator. e. Software rejects commands received out of sequence when execution of those commands out of sequence can cause a hazard. f. The software detects inadvertent memory modification and recovers to a known safe state. g. The software performs integrity checks on inputs and outputs to/from the software system. h. The software performs prerequisite checks prior to the execution of safety-critical software commands. i. No single software event or action is allowed to initiate an identified hazard. j. The software responds to an off-nominal condition within the time needed to prevent a hazardous event. k. The software provides error handling. l. The software can place the system into a safe state. | 1. Analyze the software requirements and the software design and work with the project to implement NPR 7150.2 requirement items "a" through "l." 2. Assess that the source code satisfies the conditions in the NPR 7150.2 requirement "a" through "l" for safety-critical and mission-critical software at each code inspection, test review, safety review, and project review milestone. 3. Confirm 100% code test coverage is addressed for all identified software safety-critical software components or assure that software developers provide a risk assessment explaining why the test coverage is not possible for the safety-critical code component. 4. Confirm that all identified safety-critical software components have a cyclomatic complexity value of 15 or lower. If not, assure that software developers provide a risk assessment explaining why the cyclomatic complexity value needs to be higher than 15 and why |
...
| the |
...
| software component cannot be structured to be lower than 15. 5. Confirm that the values of the safety-critical loaded data, uplinked data, rules, and scripts that affect hazardous system behavior have been tested. 6. Analyze the software design to ensure: a. Use of partitioning or isolation methods in the design and code, b. That the design logically isolates the safety-critical design elements and data from those that are non-safety-critical. 7. Participate in software reviews affecting safety-critical software products. | |||
| 3.8 | Automatic Generation of Software Source Code | ||
| 3.8.1 | 146 | The project manager shall define the approach to the automatic generation of the software source code, including: a. Validation and verification of auto-generation tools. b. Configuration management of the auto-generation tools and associated data. c. Description of the limits and the allowable scope for the use of the auto-generated software. d. Verification and validation of auto-generated source code using the same software standards and processes as hand-generated code. e. Monitoring the actual use of auto-generated source code compared to the planned use. f. Policies and procedures for making manual changes to auto-generated source code. g. Configuration management of the input to the auto-generation tool, the output of the auto-generation tool, and modifications made to the output of the auto-generation tools. | 1. Assess that the approach for the auto-generation software source code is defined, and the approach satisfies at least the conditions “a” through “g.” |
...
| 3.8.2 | 206 | The project manager shall require the software developers and suppliers to provide NASA with electronic access to the models, simulations, and associated data used as inputs for auto-generation of software. | 1. Confirm that NASA, engineering, project, software assurance, and IV&V have electronic access to the models, simulations, and associated data used as inputs for auto-generation of software. |
| 3.9 | Software Development Processes and Practices | ||
| 3.9.3 | 032 | The project manager shall acquire, develop, and maintain software from an organization with a non-expired CMMI-DEV rating as measured by a CMMI Institute Certified Lead Appraiser as follows: a. For Class A software: CMMI-DEV Maturity, Level 3 Rating, or higher for software. b. For Class B software (except Class B software on NASA Class D payloads, as defined in NPR 8705.4): CMMI-DEV Maturity Level 2 Rating or higher for software. | 1. Confirm that Class A and B software that is acquired, developed, and maintained by NASA is performed by an organization with a non-expired CMMI-DEV rating, as per the NPR 7150.2 requirement. 2. Assess potential process-related issues, findings, or risks identified from the CMMI assessment findings. 3. Perform audits on the software development and software assurances processes. |
| 3.10 | Software Reuse | ||
| 3.10.1 | 147 | The project manager shall specify reusability requirements that apply to its software development activities to enable future reuse of the software, including the models, simulations, and associated data used as inputs for auto-generation of software, for United States Government purposes |
...
| . | 1. Confirm that the project has considered reusability for its software development activities. | ||
| 3.10.2 | 148 | The project manager shall evaluate software for potential reuse by other projects across NASA and contribute reuse candidates to the NASA Internal Sharing and Reuse Software systems, however, if the project manager is a contractor then a civil servant must pre-approve all such software contributions; all software contributions should include, at a minimum, the following information: a. Software Title. b. Software Description. c. The Civil Servant Software Technical Point of Contact for the software product. d. The language or languages used to develop the software. e. Any third party code contained therein, and the record of the requisite license or permission received from the third party permitting the Government’s use, if applicable. | 1. Confirm that any project software contributed as a reuse candidate has the identified information in items “a” through “e.” |
| 3.11 | Software Cybersecurity | ||
| 3.11.2 | 156 | The project manager shall perform a software cybersecurity assessment on the software components per the Agency security policies and the project requirements, including risks posed by the use of COTS, GOTS, MOTS, OSS, or reused software components. | 1. Confirm the project has performed a software cybersecurity assessment on the software components per the Agency security policies and the project requirements, including risks posed by the use of COTS, GOTS, MOTS, OSS, or reused software components |
...
| . |
...
| 3.11.3 | 154 | The project manager shall identify cybersecurity risks, along with their mitigations, in flight and ground software systems, and plan the mitigations for these systems. | 1. Confirm that cybersecurity risks, along with their mitigations, are identified and managed. |
| 3.11.4 | 157 | The project manager shall implement protections for software systems with communications capabilities against unauthorized access. | 1. For software products with communications capabilities, confirm that the software requirements and software design documentation addresses unauthorized access. |
| 3.11.5 | 158 | The project manager shall ensure that space flight software systems are assessed for possible cybersecurity vulnerabilities and weaknesses. | 1. Confirm that space or flight software systems have been assessed for potential cybersecurity vulnerabilities and weaknesses. 2. Perform static code analysis on the software or analyze the project's static code analysis tool results for cybersecurity vulnerabilities and weaknesses. |
| 3.11.6 | 155 | The project manager shall address identified cybersecurity vulnerabilities and weaknesses. | 1. Confirm that the project addresses the engineering and assurance identified cybersecurity vulnerabilities and weaknesses. |
| 3.11.7 | 159 | The project manager shall test the software and record test results for the required software cybersecurity mitigation implementations identified from the security vulnerabilities and security weaknesses analysis. | 1. Confirm that testing is complete for the cybersecurity mitigation. 2. Assess the quality of the cybersecurity mitigation implementation testing and the test results. |
| 3.11.8 | 207 | The project manager shall identify, record, and implement secure coding practices. | 1. Assess that the software coding guidelines (e.g., coding standards) includes secure coding practices. |
| 3.11.9 | 185 | The project manager shall verify that the software code meets the project’s secure coding standard by using the results from static analysis tool(s). | 1. Analyze the engineering data or perform independent static code analysis to verify that the code meets the project’s secure coding standard requirements. |
| 3.12 | Software Bi-Directional Traceability |
...
| 3.12.1 | 052 | The project manager shall perform, record, and maintain bi-directional traceability between the following software elements:
| 1. Confirm that bi-directional traceability has been completed, recorded, and maintained. 2. Confirm that the software traceability includes traceability to any hazard that includes software |
...
| . |
...
| 4 | Software Engineering (Life-Cycle) Requirements | ||
| 4.1 | Software Requirements | ||
| 4.1.2 | 050 | The project manager shall establish, capture, record, approve, and maintain software requirements, including requirements for COTS, GOTS, MOTS, OSS, or reused software components, as part of the technical specification. | 1. Confirm that all software requirements are established, captured, and documented as part of the technical specification, including requirements for COTS, GOTS, MOTS, OSS, or reused software components. |
| 4.1.3 | 051 | The project manager shall perform software requirements analysis based on flowed-down and derived requirements from the top-level systems engineering requirements, safety and reliability analyses, and the hardware specifications and design. | 1. Perform a software assurance analysis on the detailed software requirements to analyze the software requirement sources and identify any incorrect, missing, or incomplete requirements. |
| 4.1.4 | 184 | The project manager shall include software related safety constraints, controls, mitigations, and assumptions between the hardware, operator, and software in the software requirements documentation. | 1. Analyze that the software requirements documentation contains the software related safety constraints, controls, mitigations, and assumptions between the hardware, operator, and the software. |
| 4.1.5 | 053 | The project manager shall track and manage changes to the software requirements. | 1. Confirm the software requirements changes are documented, tracked, approved, and maintained throughout the project life-cycle. |
| 4.1.6 | 054 | The project manager shall identify, initiate corrective actions, and track until closure inconsistencies among requirements, project plans, and software products. | 1. Monitor identified differences among requirements, project plans, and software products to confirm they are addressed. |
| 4.1.7 | 055 | The project manager shall perform requirements validation to ensure that the software will perform as intended in the customer environment. | 1. Confirm that the project software testing has shown that software will function as expected in the customer environment. |
| 4.2 | Software Architecture |
...
| 4.2.3 | 057 | The project manager shall transform the requirements for the software into a recorded software architecture. | 1. Assess that the software architecture addresses or contains the software structure, qualities, interfaces, and external/internal components. 2. Analyze the software architecture to assess whether software safety and mission assurance requirements are met. |
| 4.2.4 | 143 | The project manager shall perform a software architecture review on the following categories of projects: a. Category 1 Projects as defined in NPR 7120.5. b. Category 2 Projects as defined in NPR 7120.5 that have Class A or Class B payload risk classification per NPR 8705.4. | 1. Assess the results of or participate in software architecture review activities held by the project. |
| 4.3 | Software Design | ||
| 4.3.2 | 058 | The project manager shall develop, record, and maintain a software design based on the software architectural design that describes the lower-level units so that they can be coded, compiled, and tested. | 1. Assess the software design against the hardware and software requirements, and identify any gaps. 2. Assess the software design to verify that the design is consistent with the software architectural design concepts and that the software design describes the lower-level units to be coded, compiled, and tested. 3. Assess that the design does not introduce undesirable behaviors or unnecessary capabilities. 4. Confirm that the software design implements all of the required safety-critical functions and requirements. 5. Perform a software assurance design analysis. |
| 4.4 | Software Implementation |
...
| 4.4.2 | 060 | The project manager shall implement the software design into software code. | 1. Confirm that the software code implements the software designs. 2. Confirm that the code does not contain functionality not defined in the design or requirements. |
4.4.3
061
The project manager shall select and adhere to software coding methods, standards, and criteria.
1. Analyze that the software code conforms to all of the required software coding methods, rules, and principles.
4.4.4
135
The project manager shall use static analysis tools to analyze the code during the development and testing phases to detect defects, software security, and coding errors.
1. Confirm the static analysis tool(s) are used with checkers to identify security and coding errors, and defects.
2. Assess that the project addresses the results from the static analysis tools used by software assurance, software safety, engineering, or the project.
3. Confirm that the software code has been scanned for security defects and confirm the result.
4.4.5
062
The project manager shall unit test the software code.
1. Confirm that the project successfully executes the required unit tests, particularly those testing safety-critical functions.
2. Confirm that the project addresses or otherwise tracks to closure errors, defects, or problem reports found during unit test.
4.4.6
186
The project manager shall assure that the unit test results are repeatable.
1. Confirm that the project maintains the procedures, scripts, results, and data needed to repeat the unit testing (e.g., as-run scripts, test procedures, results).
4.4.7
063
The project manager shall provide a software version description for each software release.
1. Confirm that the project creates a correct software version description for each software release.
2. For each software release, confirm that the software has been scanned for security defects and coding standard compliance and confirm the results.
NASA-STD-8739.8A – 2020-06-10
35 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
4.4.8
136
The project manager shall validate and accredit the software tool(s) required to develop or maintain software.
1. Confirm that the software tool(s) needed to create and maintain software is validated and accredited.
4.5
Software Testing
4.5.2
065a
The project manager shall establish and maintain: a. Software test plan(s). …
1. Confirm that software test plans have been established, contain correct content, and are maintained.
2. Confirm that the software test plan addresses the verification of safety-critical software, specifically the off-nominal scenarios.
4.5.2
065b
The project manager shall establish and maintain: ... b. Software test procedure(s). …
1. Confirm that test procedures have been established and are updated when changes to tests or requirements occur. 2. Analyze the software test procedures for:
a. Coverage of the software requirements. b. Acceptance or pass/fail criteria, c. The inclusion of operational and off-nominal conditions, including boundary conditions, d. Requirements coverage and hazards per SWE-66 and SWE-192, respectively.
4.5.2
065c
The project manager shall establish and maintain: ... c. Software test report(s).
1. Confirm that the project creates and maintains the test reports throughout software integration and test. 2. Confirm that the project records the test report data and that the data contains the as-run test data, the test results, and required approvals.
3. Confirm that the project records all issues and discrepancies found during each test.
4. Confirm that the project tracks to closure errors, defects, etc. found during testing.
NASA-STD-8739.8A – 2020-06-10
36 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
4.5.3
066
The project manager shall test the software against its requirements.
1. Confirm test coverage of the requirements through the execution of the test procedures. 2. Perform test witnessing for safety-criticality software.
3. Ensure that any newly identified software contributions to hazards, events, or conditions found during testing are in the system safety data package.
4.5.4
187
The project manager shall place software items under configuration management prior to testing.
1. Confirm that software items to be tested are under configuration management before the start of testing.
2. Confirm the project maintains the software items under configuration management through the completion of testing.
4.5.5
068
The project manager shall evaluate test results and record the evaluation.
1. Confirm that test results are assessed and recorded.
2. Confirm that the project documents software non-conformances in a tracking system.
3. Confirm that test results are sufficient verification artifacts for the hazard reports.
4.5.6
070
The project manager shall use validated and accredited software models, simulations, and analysis tools required to perform qualification of flight software or flight equipment.
1. Confirm the software models, simulations, and analysis tools used to achieve the qualification of flight software or flight equipment have been validated and accredited.
4.5.7
071
The project manager shall update the software test plan(s) and the software test procedure(s) to be consistent with software requirements.
1. Analyze that software test plans and software test procedures cover the software requirements and provide adequate verification of hazard controls, specifically the off-nominal scenarios.
4.5.8
073
The project manager shall validate the software system on the targeted platform or high-fidelity simulation.
1. Confirm that the project validates the software components on the targeted platform or a high-fidelity simulation.
NASA-STD-8739.8A – 2020-06-10
37 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
4.5.9
189
The project manager shall ensure that the code coverage measurements for the software are selected, implemented, tracked, recorded, and reported.
1. Confirm that code coverage measurements have been selected, performed, tracked, recorded, and communicated with each release.
4.5.10
190
The project manager shall verify code coverage is measured by analysis of the results of the execution of tests.
1. Confirm that the project performs code coverage analysis using the results of the tests or by use of a code coverage tool.
2. Analyze the code coverage measurements for uncovered software code.
3. Assess any uncovered software code for potential risk, issues, or findings.
4.5.11
191
The project manager shall plan and conduct software regression testing to demonstrate that defects have not been introduced into previously integrated or tested software and have not produced a security vulnerability.
1. Confirm that the project plans regression testing and that the regression testing is adequate and includes retesting of all safety-critical code components.
2. Confirm that the project performs the planned regression testing.
3. Identify any risks and issues associated with the regression test set selection and execution.
4. Confirm that the regression test procedures are updated to incorporate tests that validate the correction of critical anomalies.
4.5.12
192
The project manager shall verify through test the software requirements that trace to a hazardous event, cause, or mitigation technique.
1. Confirm that the project verifies the software requirements, which trace to a hazardous event, cause, or mitigation techniques, through testing.
4.5.13
193
The project manager shall develop acceptance tests for loaded or uplinked data, rules, and code that affects software and software system behavior.
1. Confirm that the project develops acceptance tests for loaded or uplinked data, rules, and code that affect software and software system behavior.
2. Confirm that the loaded or uplinked data, rules, scripts, or code that affects software and software
NASA-STD-8739.8A – 2020-06-10
38 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
system behavior is baselined in the software configuration system.
4.5.14
211
The project manager shall test embedded COTS, GOTS, MOTS, OSS, or reused software components to the same level required to accept a custom developed software component for its intended use.
1. Confirm that the project is testing COTS, GOTS, MOTS, OSS, or reused software components to the same level as developed software for its intended use.
4.6
Software Operations, Maintenance, and Retirement
4.6.2
075
The project manager shall plan and implement software operations, maintenance, and retirement activities.
1. Assess the plans for maintenance, operations, and retirement for completeness of the required software engineering and software assurance activities.
2. Confirm that the project implements software operations, software maintenance, and software retirement plans.
4.6.3
077
The project manager shall complete and deliver the software product to the customer with appropriate records, including as-built records, to support the operations and maintenance phase of the software’s life-cycle.
1. Confirm that the correct version of the products is provided, including as-built documentation and project records.
2. Perform audits on the configuration management processes to verify that all products are being delivered and are the correct versions.
NASA-STD-8739.8A – 2020-06-10
39 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
4.6.4
194
The project manager shall complete, prior to delivery, verification that all software requirements identified for this delivery have been met, that all approved changes have been implemented and that all defects designated for resolution prior to delivery have been resolved.
1. Confirm that the project has identified the software requirements to be met, the approved changes to be implemented, and defects to be resolved for each delivery.
2. Confirm that the project has met all software requirements identified for the delivery.
3. Confirm that approved changes have been implemented and tested.
4. Confirm that the approved changes to be implemented and the defects to be resolved have been resolved.
5. Approve or sign-off on the projects delivered products.
4.6.5
195
The project manager shall maintain the software using standards and processes, per the applicable software classification throughout the maintenance phase.
1. Perform audits on the standards and processes used throughout maintenance based on the software classification.
4.6.6
196
The project manager shall identify the records and software tools to be archived, the location of the archive, and procedures for access to the products for software retirement or disposal.
1. Confirm that the project has identified the records and software tools for archival.
2. Confirm the project has an archival location and the procedures for archiving and accessing products for software retirement or disposal. 3. Confirm that the project archives all software and records as planned.
5
Supporting Software Life-Cycle Requirements
5.1
Software Configuration Management
NASA-STD-8739.8A – 2020-06-10
40 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
5.1.2
079
The project manager shall develop a software configuration management plan that describes the functions, responsibilities, and authority for the implementation of software configuration management for the project.
1. Assess that a software configuration management plan has been developed and complies with the requirements in NPR 7150.2 and Center/project guidance.
5.1.3
080
The project manager shall track and evaluate changes to software products.
1. Analyze proposed software and hardware changes to software products for impacts, particularly to safety and security. 2. Confirm: a. that the project tracks the changes, b. that the changes are approved and documented before implementation, c. that the implementation of changes is complete, and d. that the project tests the changes. 3. Confirm software changes follow the software change control process.
5.1.4
081
The project manager shall identify the software configuration items (e.g., software records, code, data, tools, models, scripts) and their versions to be controlled for the project.
1. Confirm that the project has identified the configuration items and their versions to be controlled.
2. Assess that the software safety-critical items are configuration managed, including hazard reports and safety analysis.
5.1.5
082
The project manager shall establish and implement procedures to: a. Designate the levels of control through which each identified software configuration item is required to pass. b. Identify the persons or groups with authority to authorize changes.
1. Confirm that software assurance has participation in software control activities.
2. Perform an audit against the configuration management procedures to confirm that the project is following the established procedures.
NASA-STD-8739.8A – 2020-06-10
41 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
c. Identify the persons or groups to make changes at each level.
5.1.6
083
The project manager shall prepare and maintain records of the configuration status of software configuration items.
1. Confirm that the project maintains records of the configuration status of the configuration items.
5.1.7
084
The project manager shall perform software configuration audits to determine the correct version of the software configuration items and verify that they conform to the records that define them.
1. Confirm that the project manager performed software configuration audits to determine the correct version of the software configuration items and verify that the results of the audit conform to the records that define them.
5.1.8
085
The project manager shall establish and implement procedures for the storage, handling, delivery, release, and maintenance of deliverable software products.
1. Confirm that the project establishes procedures for storage, processing, distribution, release, and support of deliverable software products.
2. Perform audits on the project to ensure that the project is following defined procedures for deliverable software products.
5.1.9
045
The project manager shall participate in any joint NASA/developer audits.
1. Participate in or assess the results from any joint NASA/developer audits. Track any findings to closure.
5.2
Software Risk Management
5.2.1
086
The project manager shall record, analyze, plan, track, control, and communicate all of the software risks and mitigation plans.
1. Confirm and assess that a risk management process includes recording, analyzing, planning, tracking, controlling, and communicating all of the software risks and mitigation plans.
2. Perform audits on the risk management process for the software activities.
5.3
Software Peer Reviews/Inspections
NASA-STD-8739.8A – 2020-06-10
42 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
5.3.2
087
The project manager shall perform and report the results of software peer reviews or software inspections for: a. Software requirements. b. Software plans. c. Any design items that the project identified for software peer review or software inspections according to the software development plans. d. Software code as defined in the software and or project plans. e. Software test procedures.
1. Confirm that software peer reviews are performed and reported on for project activities.
2. Confirm that the project addresses the accepted software peer review findings.
3. Perform peer reviews on software assurance and software safety plans.
4. Confirm that the source code satisfies the conditions in the NPR 7150.2 requirement SWE-134, "a" through "l," based upon the software functionality for the applicable safety-critical requirements at each code inspection/review.
5. For code peer reviews, confirm that all identified software safety-critical components have a cyclomatic complexity value of 15 or lower or develop a risk for the software safety-critical components that have a cyclomatic complexity value over 15.
5.3.3
088
The project manager shall, for each planned software peer review or software inspection: a. Use a checklist or formal reading technique (e.g., perspective based reading) to evaluate the work products. b. Use established readiness and completion criteria. c. Track actions identified in the reviews until they are resolved. d. Identify the required participants.
1. Confirm that the project meets the NPR 7150.2 criteria in "a" through "d" for each software peer review.
2. Confirm that the project resolves the actions identified from the software peer reviews.
3. Perform audits on the peer-review process.
5.3.4
089
The project manager shall, for each planned software peer review or software inspection, record necessary measurements.
1. Confirm that the project records the software peer reviews or software inspection results measurements.
NASA-STD-8739.8A – 2020-06-10
43 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
5.4
Software Measurements
5.4.2
090
The project manager shall establish, record, maintain, report, and utilize software management and technical measurements.
1. Confirm that a measurement program establishes, records, maintains, reports, and uses software assurance, management, and technical measures.
2. Perform trending and analyses on metrics (quality metrics, defect metrics) and report.
5.4.3
093
The project manager shall analyze software measurement data collected using documented project-specified and Center/organizational analysis procedures.
1. Confirm software measurement data analysis conforms to documented analysis procedures. 2. Analyze software assurance measurement data collected.
5.4.4
094
The project manager shall provide access to the software measurement data, measurement analyses, and software development status as requested to the sponsoring Mission Directorate, the NASA Chief Engineer, the Center Technical Authorities and Headquarters SMA.
1. Confirm access to software measurement data, analysis, and status as requested to the following entities: - Sponsoring Mission Directorate - NASA Chief Engineer - Center Technical Authorities - Headquarters SMA
5.4.5
199
The project manager shall monitor measures to ensure the software will meet or exceed performance and functionality requirements, including satisfying constraints.
1. Confirm the project monitors and updates planned measurements to assure the software will meet or exceed performance and functionality requirements, including satisfying constraints. 2. Monitor and track any performance or functionality requirements that are not being met or are at risk of not being met.
5.4.6
200
The project manager shall collect, track, and report software requirements volatility metrics.
1. Confirm that the project collects, tracks, and reports on the software volatility metrics. 2. Analyze software volatility measures to evaluate requirements stability as an early indicator of project problems.
NASA-STD-8739.8A – 2020-06-10
44 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
5.5
Software Non-conformance or Defect Management
5.5.1
201
The project manager shall track and maintain software non-conformances (including defects in tools and appropriate ground software).
1. Confirm that all software non-conformances are recorded and tracked to resolution.
2. Confirm that accepted non-conformances include the rationale for the non-conformance.
5.5.2
202
The project manager shall define and implement clear software severity levels for all software non-conformances (including tools, COTS, GOTS, MOTS, OSS, reused software components, and applicable ground systems).
1. Confirm that all software non-conformances severity levels are defined. 2. Assess the application and accuracy of the defined severity levels to software non-conformances.
3. Confirm that the project assigns severity levels to non-conformances associated with tools, COTS, GOTS, MOTS, OSS, reused software components, and applicable ground systems.
4. Maintain or have access to the number of software nonconformances at each severity level for each software configuration item.
5.5.3
203
The project manager shall implement mandatory assessments of reported non-conformances for all COTS, GOTS, MOTS, OSS, or reused software components.
1. Confirm the evaluations of reported non-conformances for all COTS, GOTS, MOTS, OSS, or reused software components are occurring throughout the project life-cycle. 2. Assess the impact of non-conformances on the safety, quality, and reliability of the project software.
5.5.4
204
The project manager shall implement process assessments for all high severity software non-conformances (closed loop process).
1. Perform or confirm that a root cause analysis has been completed on all identified high severity software nonconformances, the results are recorded, and that the results have been assessed for adequacy.
2. Confirm that the project analyzed the processes identified in the root cause analysis associated with the high severity software non-conformances.
NASA-STD-8739.8A – 2020-06-10
45 of 59
NPR 7150.2 Section SWE # NPR 7150.2 Requirement Software Assurance and Software Safety Tasks
3. Assess opportunities for process improvement on the processes identified in the root cause analysis associated with the high severity software non-conformances.
4. Perform or confirm tracking of corrective actions to closure on high severity software non-conformances.
...
a. Conduct an initial planning and risk assessment effort to determine the specific system/software behaviors (including the software components responsible for implementing the behaviors) to be analyzed, the IV&V tasks to be performed, the rigor to be applied to the tasks, and any tailoring of the requirements in this standard to be applied to the IV&V effort.
| Note |
|---|
| IV Note: IV&V is a focused activity that prioritizes IV&V analysis to address the highest developmental and operational software risks. IV&V priority is based on the combination of the potential for software impacts to safety and mission success and the probability factors for latent defects. IV&V analysis tasks provide coverage with a degree of rigor that reflects the priority level. The initial planning and scoping effort based on the risk assessment define the starting point for the IV&V analysis. The planning and scoping effort also aid in establishing the initial relationships between the IV&V Provider, the Acquirer, and the Provider. |
b. Develop and negotiate with the project an IV&V Execution Plan documenting the activities, methods, level of rigor, environments, tailoring (if any) of these requirements, and criteria to be used in performing verification and validation of in-scope system/software behaviors (including responsible software components) determined by the planning and scoping effort.
| Note |
|---|
Note: A provider should use a documented analysis approach to track and manage the IV&V effort in alignment with on-going development project efforts. The IV&V Project Execution Plan documents which software products will be subject to which analyses, and which analysis requirements will be fully, partially, or not applied following the risk assessment and resource constraints. The IV&V plan also serves as a communication tool between the project and IV&V to set expectations for the IV&V products produced throughout the life-cycle. |
c. Provide analysis results, risks, and assurance statements/data to all the responsible organizations’ project management, engineering, and assurance personnel.
| Note |
|---|
Note: While independent, the IV&V Provider is still a part of the overall safety and risk mitigation software assurance strategy for a project. The results of IV&V analysis need to be incorporated into the overall software assurance assessment of the project as well as provided to the project management. |
d. Participate in project reviews of software activities by providing status and results of software IV&V activities including, but not limited to, upcoming analysis tasks, artifacts needed from the project, the results of the current or completed analysis, defects, and risks to stakeholders, customers, and development project personnel.
| Note |
|---|
Note: The most significant positive impact of IV&V analysis is when the analysis results are in phase with the development effort. Communicating defects after development artifacts are baselined increases the cost to make the changes. Additionally, the inclusion of the IV&V Provider in ongoing technical meetings keeps the IV&V Provider informed of possible changes that may affect future IV&V tasking. Supporting the ongoing technical meetings allows the IV&V Provider an opportunity to provide real-time feedback on these changes. |
e. Participate in planned software peer reviews or software inspections and record peer review measurements guided by the planning and scoping risk analysis documented in the IV&V Project Execution Plan as well as by the content of the items being reviewed or inspected.
| Note |
|---|
Note: The IV&V Provider should be involved in the review/inspection process for all system/software items within the scope of their analysis. |
f. Establish, record, maintain, report, and utilize software management and technical measurements.
...
4.4.3.2 IV&V Work during Concept Development. The IV&V Provider shall verify and validate that the concept documentation represents the delineation of a specific implementation solution to solve the Acquirer’s problem. (SASS-03)
| Note |
|---|
Note: The objective of Concept IV&V is to understand the selected solution and to validate the role of software as it relates to providing the capability(ies) that support high priority or high-risk system capability(ies). |
a. Ensure that software planned for reuse meets the fit, form, and function as a component within the new application.
...
b. That the in-scope software requirements and system requirements are, at a minimum, correct, consistent, complete, accurate, readable, traceable, and testable.
| Note |
|---|
Note: Software usually provides the interface between the user and the system hardware as well as the interface between system hardware components and other systems. These interfaces are critical to the successful operation and use of the system. |
c. That the project-identified mitigations for identified security risks are in the software requirements.
| Note |
|---|
Note: Security is an essential aspect of any system development effort. In most systems, software provides the primary user interface. The user interface is an element of the system that can provide undesired access. A system concept design should address known security risks through various features in the system. |
...
a. That the relationship between the in-scope system/software requirements and the associated architectural elements is traceable correct, consistent, and complete.
| Note |
|---|
Note: Architectural elements are responsible for implementing specific behaviors within the software and the overall system. It is the interactions between these architectural elements that result in the realization of the desired behaviors as well as possible undesired behaviors. |
b. That the software architecture meets the user’s safety and mission-critical needs as defined in the requirements.
| Note |
|---|
Note: The architecture provides the foundation for the development of the software. It also significantly impacts how the software deals with faults and failures, as well as how the software interfaces with the user and system components. Analysis of the architecture provides early insight into how the software is structured and how that structure can implement the requirements. |
c. That the detailed design products are traceable, consistent, complete, accurate, and testable.
| Note |
|---|
Note: Detailed design is the implementation of the algorithms that will control and monitor the different parts of the system as well as allow for interaction between the system and the user and other systems. The detailed design defines how the architectural components will behave to support the interactions defined in the architecture. Analysis of the detailed design includes looking at the low-level software components in the software system. |
d. That the interfaces between the detailed design components and the hardware, users, operators, other software, and external systems are correct, consistent, complete, accurate, and testable.
| Note |
|---|
Note: While the architecture defines the interactions between the architectural elements, each element is generally composed of lower-level components defined by the detailed design. The interfaces between these components are important in ensuring that the architectural element meets its assigned responsibilities. |
e. That the relationship between the software requirements and the associated detailed design components is correct, consistent, and complete.
| Note |
|---|
Note: The detailed design components capture the approach to implementing the software requirements, including the requirements associated with fault management, security, and safety. Analysis of the relationship between the detailed design and the software requirements provides evidence that all of the requirements are in the detailed design. |
...
h. The source code through the use of analysis tools (including but not limited to static, dynamic, and formal analysis tools).
| Note |
|---|
Note: The use of analysis tools may include the verification and validation of the results of the analysis tools used by the development project in the process of developing the software. |
...
j. That the relationship between software code components and corresponding requirements is correct, complete, and consistent.
| Note |
|---|
Note: For all of the implementation requirements, it is with code that the development of software reaches its lowest level of abstraction and that the software capabilities are implemented. Evaluating the relationship between the source code and the design components and requirements provides evidence that only the specified requirements and components are in the system. Evaluating the relationship between the source code and the design components and requirements helps to minimize one aspect of the emergence of unexpected behaviors: inclusion of behaviors not specified in the requirements. The overall analysis of the code is essential in assuring that the code does implement the required software behaviors. From a safety perspective, it is important to evaluate the code and assure that known software safety and security issues such as buffer overflows and type mismatches, among many others, are not used in safety-critical aspects of the software. |
...
d. Verify that the software test results meet the associated acceptance criteria to ensure that software correctly implements the associated requirements.
| Note |
|---|
Note: The IV&V Provider assesses the testing artifacts within the context of the system’s meaning concerning the desired capabilities and expected operational system environment. The assessment includes an examination of testing at system boundary conditions to include unexpected conditions. The testing analysis assures that the project tests all of the requirements and that the system does what the requirements state it should do. The testing analysis also includes analysis of the traceability information between the tests and the requirements. |
e. Verify that the project tests the required software security risk mitigations to ensure that the security objectives for the software are satisfied.
...
Table 2. Additional considerations when identifying software causes in hazard analysis
| Additional Considerations | Causes | Controls |
|---|---|---|
| Computer Reset | 1. Reset with no restart 2. Reset during program upload (PROM corruption) 3. Boot PROM corruption preventing reset 4. Watchdog active during reboot causing infinite boot loop | 1. Disable FDIR and watchdogs during boot 2. Redundant computers |
| Memory | 1. Memory corruption 2. Out of Memory 3. Buffer overrun 4. Deadlock (trying to write to same memory at the same time or trying to update while reading it) 5. Single Event Upset/bit flip 6. Shared Memory (can defeat redundancy independence) | 1. No Dynamic allocation of memory after initialization 2. EDAC/Memory scrubbing 3. Memory margin |
| CPU | 1. Cycle overrun 2. Cycle jitter 3. Stack processing/ordering incorrect (e.g., FIFO vs. LIFO vs. unknown) 4. One task failing to release shared resource 5. Single Event Upset 6. Hardware failure | 1. CPU utilization and cycle time margin 2. Self-checking CPU cores |
| Commanding | 1. Inadvertent/Erroneous commanding 2. Failure to command 3. Command out of sequence 4. Command corrupted or illegitimate command= | 1. Command validation (cyclic redundancy check, timestamp, destination/source) 2. Acknowledgment/ negative acknowledgment |
| Communication and Input/Output | 1. Communication bus overload 2. Lack of or intermittent communication 3. Complex configuration input 4. Babbling Node | 1. Communication bus utilization margin 2. Use of lossless protocol 3. Downstream input voting 4. External safing function 5. Bus Guardian |
| Display Data | 1. Incorrect Data (unit conversion, incorrect variable type, etc.) 2. Stale Data | 1. Visual indication of stale data 2. Watchdog timer |
| Events and Actions | 1. Out-of-sequence event protection 2. Multiple events/actions trigger simultaneously (when not expected) 3. Error/Exception handling missing or incomplete 4. Inadvertent mode transition | 1. Fault Management 2. Pre-requisite logic 3. Interlocks |
| Timekeeping | 1. Time runs fast/slow 2. Time skips (e.g., Global Positioning System time correction) 3. Time sync across components 4. Oscillator Drift | 1. Diverse/redundant time sources with fault down logic 2. Robust time sync design that can deal with the loss of external time sources 3. Prelaunch checkout of Oscillators |
| Timing Problems | 1. Priority inversion 2. Failure to terminate/complete process in a given time 3. Data latency/sampling rate too slow 4. Race Conditions 5. Non-determinism | 1. Static and Dynamic Analysis Tools 2. Coding standards |
| Coding, Logic, and Algorithm failures | 1. Division by zero 2. Bad data in = bad data out (no parameter range & boundary checking) 3. Dead code 4. Unused code 5. Non-functional loops 6. Endless do loops 7. Incorrect passes (too many or too few or not at the correct time) 8. Incorrect “if-then” and incorrect “else.” 9. Too many or too few parameters for the called function 10. Case/type mismatch 11. Precision mismatch 12. Rounding or truncation fault 13. Resource contention (e.g., thrashing: two or more processes accessing a shared resource) 14. Bad configuration data/no checks on external input files and data 15. Inappropriate equation 16. Undefined or non-initialized data 17. Limit ranges 18. Relationship logic for interdependent limits 19. Overflow or underflow in the calculation | 1. Use of industry-accepted coding standard |
| Input Failures | 1. Noise 2. Sensors or actuators stuck at some value (all zeros, all ones, some other value) 3. A value above or below range 4. Value in range but incorrect 5. Physical units incorrect 6. Inadequate data sampling rate | 1. Sensor Health Checks 2. Input Validation, Sanity Checks |
| User Interface Errors | 1. Wrong commands are given by the operator 2. No commands are given by the operator 3. Status and messages not provided for operations, systems, and inhibits 4. Ambiguous or incorrect messages 5. User display locks up/fails | 1. Two-step commands 2. GUI style guide 3. Software interlocks to prevent human error |
| Configuration Management | 1. Incorrect version loaded 2. Incorrect configuration values | 1. Version CRC check after software/configuration load |
| Security | 1. Denial/Interruption of Service 2. Spoofed/Jammed inputs 3. An unauthorized input/access | 1. Message filtering to detect spoofing 2. Ensure software has data source validation checking features |
