1. Requirements

2.1.2.7 The NASA Chief, SMA shall make the final decision on all proposed tailoring of SWE-141, the Independent Verification and Validation (IV&V) requirement. 

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 History

2. Rationale

Independent validation and verification (IV&V) is a part of Software Assurance playing a role in the NASA software risk mitigation strategy. OSMA is responsible for determining which projects have IV&V for NASA in conjunction with the responsible Mission Directorate. The rationale for independent validation and verification (IV&V) on a project is to reduce the risk of failures due to software and provide assurance that the software will operate as intended, not operate unexpectedly and respond appropriately to adverse conditions. Performing IV&V on projects yields greater confidence that the delivered software products are error-free and meet the customer’s needs.  IV&V across the project life cycle increases the likelihood of uncovering high-risk errors early in the life cycle.

3. Guidance

The NASA IV&V Board of Advisors supports the NASA Chief, Safety and Mission Assurance by recommending significant project needs for software IV&V beyond projects meeting the criteria in items a. and b. of SWE-141 - Software Independent Verification and Validation. Exceptions to the above requirement will be written by the project and responsible Center SMA organization, adjudicated by the NASA IV&V Board of Advisors, with the final decision by the NASA Chief, Safety and Mission Assurance. Additional projects, projects in other phases, or projects without a payload risk classification can be selected by the Mission Directorate Associate Administrator to be required to have software IV&V. It is NASA policy to use the NASA Independent Verification and Validation Facility as the sole provider of IV&V services when software created by or for NASA is selected for IV&V by the NASA Chief, Safety and Mission Assurance. IV&V support is funded and managed independently of the selected project. The IV&V Advisory Board will review the scope of NASA IV&V activities on an annual basis as part of the budget planning process. The scope of IV&V services is determined by the IV&V provider, documented in the IPEP, and approved by the NASA IV&V Program. The IPEP is developed by the IV&V provider and serves as the operational document that will be shared with the project receiving IV&V support.

3.1 Independent Verification and Validation (IV&V)

IV&V is a technical discipline of software assurance that employs rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.) and off-nominal conditions (response to faults, responses to hazardous conditions, etc.). The goal of the IV&V effort is to contribute to the assurance conclusions to the project and stakeholders based on evidence found in software development artifacts and risks associated with the intended behaviors of the software. 

Three parameters define the independence of IV&V: technical independence, managerial independence, and financial independence.

1. Technical independence requires that the personnel performing the IV&V analysis are not involved in the development of the system or its elements. The IV&V team establishes an understanding of the problem and how the system addresses the problem. Through technical independence, the IV&V team’s different perspective allows it to detect subtle errors overlooked by personnel focused on developing the system.
2. Managerial independence requires that the personnel performing the IV&V analysis are not in the same organization as the development and program management team. Managerial independence also means that the IV&V team makes its own decisions about which segments of the system and its software to analyze and test, chooses the IV&V analysis methods to apply, and defines the IV&V schedule of activities. While independent from the development and program management organization, the IV&V team provides its findings in a timely manner to both of those organizations. The submission of findings to the program management organization should not include any restrictions (e.g., requiring the approval of the development organization) or any other adverse pressures from the development group.
3. Financial independence requires that the control of the IV&V budget be vested in a group independent of the software development organization. Financial independence does not necessarily mean that the IV&V team controls the budget but that the finances should be structured so that funding is available for the IV&V team to complete its analysis or test work. No adverse financial pressure or influence is applied.

3.2 IV&V Project Involvement

The IV&V process starts early in the software development life cycle, providing feedback to the Provider organization, allowing them to modify products at optimal timeframes and in a timely fashion, thereby reducing overall project risk. The feedback also answers project stakeholders’ questions about system properties (correctness, robustness, safety, security, etc.) to make informed decisions with respect to the development and acceptance of the system and its software.

The IV&V Provider performs two primary activities, often concurrently: verification and validation. Each of the activities provides a different perspective on the system/software. Verification is the process of evaluating a system and its software to provide objective evidence as to whether or not a product conforms to the build-to requirements and design specifications. Verification holds from the requirements through the design and code and into testing. Verification seeks to demonstrate that the products of a given development phase satisfy the conditions imposed at the start of or during that phase. Validation tasks seek to develop objective evidence that shows that the content of the engineering artifact is the right content for the developed system/software. The content is accurate and correct if the objective evidence demonstrates that it satisfies the system requirements (e.g., user needs, stakeholder needs, etc.), fully describes the required capability/functionality needed, and solves the right problem.

3.3 Objective Evidence

The center of the IV&V effort is on identifying and generating objective evidence that supports the correct operation of the system or refutes the correct operation of the system. The IV&V Provider typically works with the development team to understand this objective evidence, which provides artifacts such as concept studies, operations concepts, and requirements that define the overall project. The IV&V Provider uses these materials to develop an independent understanding of the project’s commitment to NASA, which forms the basis for validating lower-level technical artifacts.

Two principles help guide the development and use of objective evidence.

1. Performing IV&V throughout the entire development lifetime is the first principle; potential problems should be detected as early as possible in the development life cycle. Performing IV&V throughout the entire development lifetime provides the IV&V team with sufficient information to establish a basis for the analysis results and provides early objective evidence to the development and program management groups to help keep the development effort on track early in the life cycle.
2. The second principle is “appropriate assurance.” Given that it is not possible to provide IV&V on all aspects of a project’s software, the IV&V Provider and project should balance risks against available resources to define an IV&V program for each project that provides IV&V so that the software will operate correctly, safely, reliably, and securely throughout its operational lifetime. The IV&V Project Execution Plan documents this tailored approach and summarizes the cost/benefit trade-offs made in the scoping process.

3.4 IV&V Requirements

The IV&V requirements are analyzed and partitioned according to the type of artifact. The requirements do not imply or require the use of any specific life cycle model. It is also important to understand that IV&V applies to any life cycle development process. The IV&V requirements document the potential scope of analysis performed by the IV&V Provider and the key responsibility of the software project to provide the information needed to perform that analysis. Additionally, scoping the IV&V analysis is according to the application of the risk assessment to determine the prioritization of activities and the level of rigor associated with performing those activities. 

3.5 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

3.6 Center Process Asset Libraries

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only). 

4. Small Projects

No additional guidance is available for small projects.

5. Resources

5.1 References

5.2 Tools

6. Lessons Learned

6.1 NASA Lessons Learned

The NASA Lessons Learned database contains the following lessons learned related to IV&V:

• Independent Verification and Validation of Embedded Software (Use of IV&V Procedures). Lesson Number 723  ⁵¹⁸: "The use of Independent Verification and Validation (IV&V) processes ensures that computer software is developed in accordance with original specifications, that the software performs the functions satisfactorily in the operational mission environment for which it was designed, and that it does not perform unintended functions. Identification and correction of errors early in the development cycle are less costly than identification and correction of errors in later phases, and the quality and reliability of software are significantly improved."

• Does Software IV&V Provide Clear Benefits to NASA Projects?  Lesson Number 6656  ⁵⁸⁴: “Recent NASA spaceflight projects that have undergone IV&V of their mission software perceive the process as offering concrete benefits beyond those accrued through VV performed solely by project personnel. The specific benefits accrued to four recent JPL spaceflight projects from participation by the NASA IVV Center are discussed, and some recommendations for future IVV programs are proposed.”

6.2 Other Lessons Learned

• Software IV&V
  • IV&V approaches and resources must align with the program risk posture.
  • A closed-loop, auditable, corrective action toolset and process must be used to manage all IV&V identified issues and risks.