3.6.2 For projects reaching KDP (Key Decision Point) A after the effective date of this directive’s revision, the program manager shall ensure that software IV&V (Independent Verification and Validation) is performed on the following categories of projects:
a. Category 1 Projects as defined in NPR 7120.5.
b. Category 2 Projects as defined in NPR 7120.5 that have Class A or Class B payload risk classification per NPR 8705.4.
c. Projects specifically selected by the NASA CSMA to have software IV&V.
The NASA Independent Verification and Validation (IV&V) Board of Advisors supports the NASA Chief, Safety and Mission Assurance (CSMA) by recommending significant project needs for software IV&V beyond projects meeting the criteria in items a. & b. of SWE-141. Waivers to the above requirement will be written by the project and responsible Center S&MA organization, adjudicated by the NASA IV&V Board of Advisors, with the final decision by the NASA CSMA. Additional projects, projects in other phases, or projects without a payload risk classification can be selected by the NASA CSMA to be required to have software IV&V. It is NASA policy to use the NASA IV&V Facility as the sole provider of IV&V services when software created by or for NASA is selected for IV&V by the NASA CSMA. IV&V support is funded and managed independent of the selected project.
1.2 Applicability Across Classes
This requirement applies based on the selection criteria defined in this requirement.
The rationale for independent validation and verification (IV&V) on a project is to reduce risk of failures due to software. Performing IV&V on projects yields a greater confidence that the delivered software products are error free and meet the customer’s needs. IV&V across the project lifecycle increases the likelihood of uncovering high risk errors early in the life cycle.
Independent validation and verification (IV&V) is a part of Software Assurance playing a role in the NASA software risk mitigation strategy. IV&V is an objective examination of safety and mission critical software processes and products. The key parameters for independence are: technical independence, managerial independence and financial independence. The goal of IV&V is to determine if the right system has been built and that it has been built correctly. From that perspective IV&V strives to answer the questions: will the system’s software do what it is supposed to do, will the system’s software not do what it is not supposed to do and will the system’s software respond as expected under adverse conditions.
IV&V leads to higher quality products, reduced risk, greater insight, reduced cost and knowledge transfer. IV&V also facilitates the transfer of system and software engineering best practices. IV&V status reports provide another status indicator and performance report to decision makers (program managers).
In 2012, the Aerospace Safety Advisory Panel recommended that “NASA should establish a standard identifying the level of criticality that requires software IV&V, i.e., at what risk level must IV&V be required and therefore either be resourced, or if that is not possible, a formal waiver process be in place for an accountable individual to accept the associated risk and document it.”
The Agency concurred and directed the Office of the Chief Engineer, with assistance from the Office of Safety and Mission Assurance and the NASA IV&V Program, to develop a NASA Interim Directive (NID), now incorporated into NPR 7150.2, that would replace the existing multi-step process for determining which projects must have software IV&V with a standard identifying the level of criticality that requires software IV&V.
NPD 7120.4D, NASA Engineering and Program/Project Management Policy, reiterates the point made in the note associated with SWE-141 that, “It is NASA policy to use the NASA Independent Verification and Validation (IV&V) Facility as the sole provider of IV&V services when software created by or for NASA is selected for IV&V by the NASA Chief, Safety and Mission Assurance.”
The earlier IV&V is involved in a project, the better the provided service and value of the results will be given the additional time to understand the project, its risks, and safety-critical aspects.
Projects meeting the criteria in this requirement that are not chosen for NASA IV&V Program services due to funding limitations are expected to either independently fund the IV&V services or obtain a waiver (see SWE-126).
Expanded criteria definitions
Category 1 projects are defined in NPR 7120.5E, NASA Space Flight Program and Project Management Requirements, as human space flight projects, projects with life-cycle cost exceeding $1B, or projects with significant radioactive material.
Category 2 projects are defined in NPR 7120.5E as projects that have life-cycle costs greater than $250M and less than $1B or have life-cycle costs less than $250M with a high priority level based on “the importance of the activity to NASA, the extent of international participation (or joint effort with other government agencies), the degree of uncertainty surrounding the application of new or untested technologies” and a Class A or Class B payload risk classification.
Class A payload risk classifications are defined in NPR 8705.4, Risk Classification for NASA Payloads, as payloads with high priority, very low (minimized) risk, very high national significance, very high to high complexity, greater than 5 year mission lifetime, high cost, critical launch constraints, no alternative or re-flight opportunities, and/or payloads where “all practical measures are taken to achieve minimum risk to mission success. The highest assurance standards are used.”
Class B payload risk classifications are defined in NPR 8705.4 as payloads with high priority, low risk, high national significance, high to medium complexity, 2- 5 year mission lifetime, high to medium cost, medium launch constraints, infeasible or difficult in-flight maintenance, few or no alternative or re-flight opportunities, and/or payloads where “stringent assurance standards [are applied] with only minor compromises in application to maintain a low risk to mission success.”
Per NPR 8705.4, “The importance weighting assigned to each consideration is at the discretion of the responsible Mission Directorate.”
Additional guidance related to IV&V may be found in the following related requirement in this Handbook.:
All projects are assessed against these criteria to determine whether they should receive IV&V services.
6. Lessons Learned
The NASA Lessons Learned database contains the following lessons learned related to IV&V:
Independent Verification and Validation of Embedded Software (Use of IV&V Procedures). Lesson Number 723: "The use of Independent Verification and Validation (IV&V) processes ensures that computer software is developed in accordance with original specifications, that the software performs the functions satisfactorily in the operational mission environment for which it was designed, and that it does not perform unintended functions. Identification and correction of errors early in the development cycle are less costly than identification and correction of errors in later phases, and the quality and reliability of software are significantly improved."
Does Software IV&V Provide Clear Benefits to NASA Projects? Lesson Number 6656: “Recent NASA spaceflight projects that have undergone IV&V of their mission software perceive the process as offering concrete benefits beyond those accrued through VV performed solely by project personnel. The specific benefits accrued to four recent JPL spaceflight projects from participation by the NASA IVV Center are discussed, and some recommendations for future IVV programs are proposed.”