Software Assurance and Software Safety Requirements

The responsible project manager shall ensure the performance of the software assurance, software safety, and IV&V activities, the applicable requirements are defined in Table 1. In this document, the phrase “Software Assurance and Software Safety Tasks” means that the roles and responsibilities for completing these requirements may be delegated within the project consistent with the scope and scale of the project. The Center SMA Director designates SMA TA(s) for programs, facilities, and projects, providing direction, functional oversight, and assessment for all Agency software assurance, software safety, and IV&V

Software Assurance and Software Safety Requirements Mapping Matrix

3. Independent Verification & Validation Requirements

3.1 IV&V Overview

IV&V is a technical discipline of software assurance that employs rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the software development life The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.) and off-nominal conditions (response to faults, responses to hazardous conditions, etc.). The goal of the IV&V effort is to contribute assurance conclusions provided to the project and stakeholders based on evidence found in software development artifacts and risks associated with the intended behaviors of the software.

Three parameters define the independence of IV&V: technical independence, managerial independence, and financial independence.

1. Technical independence requires that the personnel performing the IV&V analysis are not involved in the development of the system or its elements. The IV&V team establishes an understanding of the problem and how the system addresses the problem. Through technical independence, the IV&V team’s different perspective allows it to detect subtle errors overlooked by personnel focused on developing the system.
2. Managerial independence requires that the personnel performing the IV&V analysis are not in the same organization as the development and program management team. Managerial independence also means that the IV&V team makes its own decisions about which segments of the system and its software to analyze and test, chooses the IV&V analysis methods to apply, and defines the IV&V schedule of activities. While independent from the development and program management organization, the IV&V team provides its findings in a timely manner to both of those organizations. The submission of findings to the program management organization should not include any restrictions (e.g., requiring the approval of the development organization) or any other adverse pressures from the development group.
3. Financial independence requires that the control of the IV&V budget be vested in a group independent of the software development organization. Financial independence does not necessarily mean that the IV&V team controls the budget but that the finances should be structured so that funding is available for the IV&V team to complete its analysis or test work. No adverse financial pressure or influence is applied.

The IV&V process starts early in the software development life cycle, providing feedback to the IV&V provider organization, allowing the IV&V team to modify products at optimal timeframes and in a timely fashion, thereby reducing overall project risk. The feedback also answers project stakeholders’ questions about system properties (correctness, robustness, safety, security, etc.) to make informed decisions with respect to the development and acceptance of the system and its software.

The IV&V provider performs two primary activities, often concurrently: verification and validation. Each of the activities provides a different perspective on the system/software.

1. Verification is the process of evaluating a system and its software to provide objective evidence as to whether or not a product conforms to the build-to requirements and design specifications. Verification holds from the requirements through the design and code and into testing. Verification demonstrates that the products of a given development phase satisfy the conditions imposed at the start of or during that phase.
2. Validation develops objective evidence that shows that the content of the engineering artifact is the right content for the developed system/software.

The content is accurate and correct if the objective evidence demonstrates that it satisfies the system requirements (e.g., user needs, stakeholder needs, etc.), fully describes the required capability/functionality needed, and solves the right problem.

The main goal of the IV&V effort is to identify and generate objective evidence that supports the correct operation of the system or refutes the correct operation of the system. The IV&V provider typically works with the development team to understand this objective evidence, which provides artifacts such as concept studies, operations concepts, and requirements that define the overall project. The IV&V provider uses these materials to develop an independent understanding of the project’s commitment to NASA, which forms the basis for validating lower-level technical artifacts.

Two principles help guide the development and use of objective evidence.

1. Performing IV&V throughout the entire development lifetime is the first principle; potential problems should be detected as early as possible in the development life Performing IV&V throughout the entire development lifetime provides the IV&V team with sufficient information to establish a basis for the analysis results and provides early objective evidence to the development and program management groups to help keep the development effort on track early in the life cycle.
2. The second principle is “appropriate assurance.” Given that it is not possible to provide IV&V on all aspects of a project’s software, the IV&V provider and project should balance risks against available resources to define an IV&V program for each project that provides IV&V so that the software will operate correctly, safely, reliably, and securely throughout its operational lifetime. The IPEP documents this tailored approach and summarizes the cost/benefit trade-offs made in the scoping process.

The IV&V requirements are analyzed and partitioned according to the type of artifact. The requirements do not imply or require the use of any specific life cycle model. It is also important to understand that IV&V applies to any life cycle development process. The IV&V requirements document the potential scope of analysis performed by the IV&V provider and the key responsibility of the software project to provide the information needed to perform that analysis. Additionally, the risk assessment is used to scope the IV&V analysis to help determine the prioritization of activities and the level of rigor associated with performing those activities. The scoping exercise results are captured in the IV&V Project Execution Plan, as documented below.

3.2 IV&V Requirements

The responsible project manager shall ensure the performance of the IV&V requirements, as defined in section 4.4.2 of this standard. The IV&V requirements in this section of the standard apply to any project required to have IV&V per the criteria defined in the NASA Software Engineering Requirements, NPR 7150.2. The IV&V requirements apply to all IV&V efforts performed on a software development project, as tailored by the IV&V Project Execution Plan. The IV&V requirements also serve as the definition of what NASA considers IV&V. IV&V is a risk mitigation activity, and as such, the application of IV&V analysis and the rigor of that analysis is driven by the IV&V provider’s assessment of software risk.

4.4.2.1 The IV&V provider shall conduct planning and risk assessments to determine the specific system/software behaviors (including the software components responsible for implementing the behaviors) to be analyzed.

4.4.2.2 The IV&V provider shall develop and negotiate an IV&V IPEP with the project.

4.4.2.3 The Project SMA Technical Authority (TA) shall review and concur with the IPEP.

4.4.2.4 The IV&V provider shall provide analysis results, risks, and assurance statements and data to the responsible organizations’ project management, engineering, and software assurance personnel.

4.4.2.5 The IV&V provider shall participate in project reviews of software activities. Participation includes providing status and results of software IV&V activities including, but not limited to, upcoming analysis activities, artifacts needed from the project, the results of the current or completed analysis, defects, and risks to stakeholders, customers, and development project personnel.

4.4.2.6 The IV&V provider shall provide the responsible organizations’ project management, engineering, and software assurance personnel insight into the software IV&V and IV&V test activities. As a minimum, the IV&V provider will be required to allow the responsible organizations’ project management, engineering, and software assurance personnel to perform the following activities:

1. Monitor the IV&V activities and plans.
2. Review the verification activities to ensure adequacy.
3. Review IV&V studies and source data.
4. Audit the software IV&V processes and practices.
5. Participate in IV&V software reviews and technical interchange meetings

4.4.2.7 The IV&V provider shall participate in planned software peer reviews or software inspections guided by the planning and scoping risk analysis documented in the IPEP and NASA-HDBK-2203.

4.4.2.8 The IV&V provider shall establish, record, maintain, report, and utilize IV&V management and technical measurements.

4.4.2.9 The IV&V provider shall assess and track software activities' actual results and performance against the software plans and identify and report any risks or findings to the responsible organizations’ project management, engineering, and software assurance personnel.

4.4.2.10 The IV&V provider shall track and evaluate changes to software products to evaluate for possible changes in the IV&V provider’s risk analysis and potential adverse impacts to the software system and the development effort.

4.4.2.11 The IV&V provider shall assess the software development life cycle for suitability for the problem to be solved and identify and communicate any risks associated with the chosen life cycle to the responsible organizations’ project management, engineering, and software assurance personnel.

4.4.2.12 The IV&V provider shall identify, analyze, track, and record risks to the software and development project in accordance with NPR 8000.4, Agency Risk Management Procedural Requirements, and communicate the risks to the responsible organizations’ project management, engineering, and software assurance personnel.

4.4.2.13 The IV&V provider shall verify the project implements the requirements for software listed in NPR 7150.2 and communicate any risks to the responsible organizations’ project management, engineering, and software assurance personnel.

4.4.2.14 The IV&V provider shall track, record, and communicate defects/issues and other results found during the execution of IV&V analysis and independent IV&V testing to the responsible organizations’ project management, engineering, and software assurance personnel.

4.4.2.15 The IV&V provider shall ensure that the identified defects and issues are addressed by the project.

4.4.2.16 The IV&V provider shall ensure that software planned for reuse meets the fit, form, and function as a component within the new application.

4.4.2.17 The IV&V provider shall ensure that the system architecture contains the computing-related items (subsystems, components, etc.) to carry out the system's mission and satisfy user needs and operational scenarios or use cases.

4.4.2.18 The IV&V provider shall ensure that the basis for the computing-related functions reflects the planned operations and mission objectives.

4.4.2.19 The IV&V provider shall ensure that feasibility and trade studies provide the results to support the critical decisions that drove the need for the study.

4.4.2.20 The IV&V provider shall ensure that known software-based hazard causes, contributors, and controls are identified, documented, and traced to the project requirements.

4.4.2.21 The IV&V provider shall ensure that known security threats and risks are identified, appropriately documented, and updated throughout the software development life cycle and communicated to the responsible organizations’ project management, engineering, and software assurance personnel.

4.4.2.22 The IV&V provider shall verify and validate that the software requirements and system requirements are, as a minimum, correct, consistent, complete, accurate, readable, traceable, and testable.

4.4.2.23 The IV&V provider shall verify and validate that the mitigations for identified security risks are in the software requirements.

4.4.2.24 The IV&V provider shall ensure that software requirements meet the dependability and fault tolerance required by the system.

4.4.2.25 The IV&V provider shall ensure that software requirements provide the capability of controlling identified hazards and do not create hazardous conditions.

4.4.2.26 The IV&V provider shall verify and validate that the relationship between the in-scope system/software requirements and the associated architectural elements is traceable, correct, consistent, and complete.

4.4.2.27 The IV&V provider shall verify and validate that the software architecture meets the user’s safety and mission-critical needs as defined in the requirements.

4.4.2.28 The IV&V provider shall verify and validate that the detailed design products are traceable, consistent, complete, accurate, and testable.

4.4.2.29 The IV&V provider shall verify and validate that the interfaces between the detailed design components and the hardware, users, operators, other software, and external systems are correct, consistent, complete, accurate, and testable.

4.4.2.30 The IV&V provider shall verify and validate that the relationship between the software requirements and the associated detailed design components is correct, consistent, and complete.

4.4.2.31 The IV&V provider shall verify and validate that the software code and data products are consistent with architecture, complete with respect to requirements, and testable.

4.4.2.32 The IV&V provider shall verify and validate that the software code meets the industry best practices and software coding standards.

4.4.2.33 The IV&V provider shall verify and validate that the security risks in the software code are identified and mitigated.

4.4.2.34 The IV&V provider shall verify and validate the appropriate use of off-the-shelf software, including ensuring that the project has identified all OSS used and that the security risks are identified and mitigated by the use of the off-the-shelf

4.4.2.35 The IV&V provider shall verify and validate that the project assesses the software systems for possible security vulnerabilities and weaknesses.

4.4.2.36 The IV&V provider shall verify and validate that the project implements the required software security risk mitigations to ensure that security objectives for software are satisfied.

4.4.2.37 The IV&V provider shall verify and validate the source code through the use of analysis tools (including but not limited to static, dynamic, composition, and formal analysis tools).

4.4.2.38 The IV&V provider shall verify and validate that the relationship between the software design elements and the associated software units is correct, consistent, and complete.

4.4.2.39 The IV&V provider shall verify and validate that the relationship between software code components and corresponding requirements is correct, complete, and consistent.

4.4.2.40 The IV&V provider shall verify and validate that test plans, test procedures, test cases, test environment (including simulations), and test design at all levels of testing (unit, integration, system, acceptance, etc.) are correct, complete, and consistent for verification and validation of the source code and system functions allocated to the software.

4.4.2.41 The IV&V provider shall verify and validate the relationships between the test plans, test procedures, test cases, test design, source code. and system functions allocated to the software are correct, complete, and consistent.

4.4.2.42 The IV&V provider shall verify that the test plans, test cases, test design, and test procedures contain objective acceptance criteria that support the verification of the associated requirements for both nominal and off-nominal conditions.

4.4.2.43 The IV&V provider shall verify that the software test results meet the associated acceptance criteria to ensure that the software correctly implements the associated requirements.

4.4.2.44 The IV&V provider shall verify that the project tests the required software security risk mitigations to ensure that the security objectives for the software are satisfied.

4.4.2.45 The IV&V provider shall verify that code coverage is measured by analysis of the results of the execution of tests.

4.4.2.46 The IV&V provider shall verify through independent testing each of the software requirements that trace to a hazardous event, cause, or mitigation technique.

4.4.2.47 The IV&V provider shall verify the project’s acceptance tests for loaded or uplinked data, rules, and code that affects software and software system behavior.

4.4.2.48 The IV&V provider shall participate in all NASA quality audits, assessments, and reviews associated with the project.

4.4.2.49 The IV&V provider shall assess the software maintenance and operational risks concerning software elements to support the planning of IV&V activities during the maintenance phase.