2. Rationale

The rationale for IV&V on a project is to reduce risk of failures due to software. IV&V Project Execution Plans (IPEP) describe the activities and processes that will be carried out and the products that will be produced to fulfill the project’s IV&V requirements for software. This plan is created to guide the work and increase the expectations of meeting project objectives and goals. 

3. Guidance

The purpose of the Independent Verification and Validation (IV&V) Project Execution Plan (IPEP) is two-fold. First, it is to communicate IV&V interactions, interfaces, roles and responsibilities, technical products, and reporting methods with the project.  Second, the IPEP serves as the operational document for the IV&V efforts.  The IPEP provides a mechanism for the IV&V provider, including the IV&V Program, to coordinate with a project to identify and describe IV&V activities that apply. Specifically, it includes the basic tenets for an agreement between the IV&V team and the project, including the roles and responsibilities, communications paths, and artifacts anticipated to be shared between the organizations.

The NASA IPEP template ⁰²⁸ states that the “ IPEP is prepared and maintained by the IV&V Project Manager (PM). The IV&V PM coordinates the creation and maintenance of this document with affected individuals and organizations (within the NASA IV&V Program as well as with the Project).”

Because the Independent Verification and Validation (IV&V) provider is responsible for developing the IV&V Project Execution Plan (IPEP), this guidance will address coordination efforts needed from projects receiving IV&V to support the development and execution of the IPEP.  Note that both an IPEP and a separate Memorandum of Understanding (MOU) may be established for projects receiving IV&V services, but only if the project requests the MOU.

Note that the IPEP is reviewed when it is initially developed, not at any particular life-cycle milestone review.  The milestone review at which the IPEP may be reviewed depends on which milestone the mission is approaching when the IV&V provider gets involved with the mission.  Per the IV&V Program, IPEPs are reviewed every 6-months for applicability or when there are significant changes made on the mission (e.g., a milestone slips out several months) or there is a significant change to IV&V (e.g., an budget cut which forces an IV&V provider to reduce their work on a mission). 

Software Provider Responsibilities

The responsibilities below and the means for carrying them out are documented as part of the IPEP (or other appropriate documents, in the case of contracted software efforts).

Per NASA-STD-8739.8: ²⁷⁸

• ”When IV&V has been selected for a project, the provider shall coordinate with IV&V personnel to share data and information.”
• ”When the IV&V function is required, the provider shall provide all required information to NASA IV&V Facility personnel. (This requirement includes specifying on the contracts and subcontracts, IV&V’s access to system and software products and personnel.)” This statement applies to the IV&V provider, regardless of whether that provider is the IV&V Program or another organization.
• ”The NASA IV&V Facility shall initially conduct a planning and scoping exercise to determine the specific software components to be analyzed and the tasks to be performed. The IV&V approach will be documented in an IV&V plan.” This statement is also a recommended practice for IV&V providers other than the IV&V Program.
• ”The IV&V team shall provide input to the appropriate software assurance personnel, as well as provide feedback to the project manager as agreed in the IV&V Plan.” (This helps to ensure that the IV&V team and the project’s software assurance team maintain an active and productive relationship.  Specific details of these interactions and the information to be exchanged for each project may be documented in the IPEP.)

Per NASA-STD-8739.8,  ²⁷⁸ the Center SMA organizations are assumed to carry out the following:

• “Assure that if IV&V is required on a program, project, or facility, project risk and software criticality determinations are shared between the safety personnel and IV&V.”
• “If this project is a candidate for IV&V, the Software Safety Plan shall address, either specifically or by reference to the IV&V Memorandum of Agreement, the role of IV&V for the safety-critical software and detail how IV&V will work with the software safety program and personnel.”

NASA-GB-8719.13, NASA Software Safety Guidebook, ²⁷⁶ states that for contractor-developed software, “At a minimum, management, software development, and software assurance will need to work with the IV&V or IA personnel, providing information, answers, and understanding of the software system and the development process.

“Depending on the level of IV&V levied on the project, some independent tests may be performed. These may require some contractor resources to implement.”

IV&V Responsibilities

The activities to be carried out by the IV&V provider are coordinated with the appropriate stakeholders, including the project manager. The IV&V provider captures the following types of information in the IV&V Plan:

• Goals and objectives.
• High-level description of the IV&V approach.
• IV&V activities to be performed for the project.
• IV&V deliverables.
• Milestones.
• IV&V schedule coordinated with project management reviews and technical reviews.
• Interface among IV&V, Mission Project office, and software developer.
• Data exchange requirements.
• IV&V access to appropriate artifact management systems, issue and risk tracking systems, etc.
• Need for a heritage review to identify relevant products and artifacts from previous projects.
• Heritage review results.
• Preliminary list of processes and products to be evaluated.
• IV&V access rights to proprietary and classified information; related process to obtain those rights.

It is recommended that the scoping of the IV&V effort be driven by risk, or be risk-based given that IV&V of the full software subsystem is not likely to be feasible within financial, schedule, or resource constraints. 

The IV&V provider is expected to capture detailed fiscal year activity information in the appendices as activities, risks and risk assessment results, schedules, IV&V coverage, and required resources may be subject to change over time based on project life-cycle progress and other factors.

The following actions, summarized from NASA-GB-8719.13  ²⁷⁶ and other sources may be performed (this is not an all-inclusive list):

• IV&V – IV&V analysts conduct the in-depth analyses and verifications.
• Attend project reviews – summarize IV&V findings for software products.
• Integrated addition to software QA – not a replacement.
• May perform specific safety analyses – not a replacement for the safety role.
• May perform some software engineering functions – e.g., systems engineering trade studies, such as studies used to determine if software architectures can satisfy performance needs of the system.
• Verify and validate concept documentation (the description of a specific implementation solution).
• Validate requirements and testing.
• Verify software architectures and designs.
• Verify requirements and design.
• Perform independent system testing.
• Ensure consistency from requirements to testing.
• Verify and validate test documentation.
• Verify and validate implementation.
• Verify and validate operations and maintenance content.
• Capture and report findings – tracking them to closure.

Execution Issues

Issues inherent in performing IV&V include the following for which projects and the IV&V provider coordinate to minimize the impact to project success while assuring safety, reducing risk, and providing project insight:

• IV&V "effort needs to be allocated at the right points in the development of a product (e.g. a document), so that the product is mature enough to be analyzed, but not so mature that it cannot be changed." ¹⁸⁰
• "There is always a delay between the delivery of an interim product to the IV&V team, and the completion of analysis of that product. During this time, the development process continues." ¹⁸⁰
• "...the IV&V contractor has less access to the development team than is ideal." ¹⁸⁰
• "Documentation from the development team is usually made available to ... IV&V ... in draft form... The drawback is that documents may be revised while the IV&V team is analyzing them, making the results of the analysis irrelevant before it is finished." ¹⁸⁰
• "Often, problems found by IV&V have cost and schedule implications..." ¹⁸⁰(Issues found in development products are expected to be fixed, which takes time and resources. Therefore, it is important that IV&V can work in phase with the development project so that errors in the system can be resolved efficiently.)

Table of Contents for a Typical IPEP

1          Introduction
1.1       Document Organization
1.2       Document Purpose
1.3       Intended Audience

 2          IV&V Overview
2.1       IV&V Goals and Objectives
2.2       IV&V Approach
2.2.1    Verification and Validation
2.3       IV&V Focus

3          Roles, Responsibilities and Interfaces
3.1       IV&V Program
3.1.1    Research Support
3.1.2    IV&V Metrics Support
3.2       IV&V Team
3.3       Project Personnel

4          IV&V Products and Communication/Reporting Methods
4.1       IV&V Products
4.1.1    Analysis Reports
4.1.2    Lifecycle Review Presentations
4.1.3    Issues
4.1.4    Risks
4.1.5    Item Tracking/Monitoring and Escalation
4.2       IV&V Communication and Reporting Methods
4.2.1    Lifecycle Review Presentations
4.2.2    Agency/Mission Directorate/Center Management Briefings
4.2.3    Routine Tag-ups

Appendix A:  IV&V Portfolio Based Risk Assessment (PBRA)/Risk Based Assessment (RBA) Results
Appendix B:  IV&V Focus
Appendix D:  Technical Scope & Rigor (TS&R)
Appendix E:  Reference Documentation  
Appendix F:  Acronyms
Appendix G:  Fiscal Year <XX> IV&V Efforts
 

Additional guidance related to IV&V may be found in the following related requirement in this Handbook:

4. Small Projects

All projects with mission- or safety-critical software and receiving IV&V services, are treated the same.  The same process is used for developing an IPEP, regardless of project size, and IPEPs all have the same basic elements (objectives, schedules, interfaces, etc.).  The actual content of each IPEP is, of course, different for each project.  

5. Resources

5.1 Tools

Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN).

NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.

The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.

6. Lessons Learned

The NASA Lessons Learned database contains the following lesson learned related to the IV&V plan:

• Independent Verification and Validation of Embedded Software (Use of IV&V Procedures). Lesson Number 723: "The use of Independent Verification and Validation (IV&V) processes ensures that computer software is developed in accordance with original specifications, that the software performs the functions satisfactorily in the operational mission environment for which it was designed, and that it does not perform unintended functions. Identification and correction of errors early in the development cycle are less costly than identification and correction of errors in later phases, and the quality and reliability of software are significantly improved." ⁵¹⁸