Measurement System Identification: Not Measurement Sensitive
NASA TECHNICAL STANDARD
National Aeronautics and Space Administration
NASA-STD-8739.8B
Approved: TBD Superseding NASA-STD-8739.8A
SOFTWARE ASSURANCE AND SOFTWARE SAFETY STANDARD APPROVED FOR PUBLIC RELEASE – DISTRIBUTION IS UNLIMITED
DOCUMENT HISTORY LOG
Include Page
SWE-057 - SA Task1
SWE-057 - SA Task1
Status
Document Revision
Approval Date
Description
Baseline
Initial
2004-07-28
Initial Release
1
2005-05-05
Administrative changes to the Preface; Paragraphs 1.1, 1.4, 1.5, 2.1.1, 2.2.2, 3, 5.1.2.3, 5.4.1.1; 5.6.2, 5.8.1.2, 6.7.1.a, 7.3.2, 7.3.3, 7.5, 7.5.1; Table 1; Appendix A; Appendix C to reflect NASA Transformation changes, reflect the release of NASA Procedural Requirements (NPR) 7150.2, NASA Software Engineering Requirements and to make minor editorial changes. Note: Some paragraphs have changed pages as a result of these changes. Only pages where content has changed are identified by change indications.
A
2020-06-10
The revised document addresses the following significant issues: combined the NASA Software Assurance Standard (NASA-STD-8739.8) with the NASA Software Safety Standard (NASA-STD-8719.13), reduction of requirements, bring into alignment with updates to NPR 7150.2, added a section on IV&V requirements to perform IV&V, and moved guidance text to an Electronic Handbook. This change combines the updates to NASA-STD-8739.8 and the content of NASA-STD-8719.13. The update includes the NASA software safety requirements and cancels NASA-STD-8719.13 standard.
B
TBD
Brings into alignment with the update to NPR 7150.2D. Update the Appendix A table containing the additional areas to consider when identifying software causes in Hazard Analysis.
Forward
This NASA Technical Standard is published by the National Aeronautics and Space Administration (NASA) to provide uniform engineering and technical requirements for processes, procedures, practices, and methods that have been endorsed as standard for NASA facilities, programs, and projects, including requirements for selection, application, and design criteria of an item. This standard was developed by the NASA Office of Safety and Mission Assurance (OSMA). Requests for information, corrections, or additions to this standard should be submitted to the OSMA by email to Agency-SMA-Policy-Feedback@mail.nasa.gov or via the “Email Feedback” link at https://standards.nasa.gov.
Russ Deloach
NASA Chief, Safety and Mission Assurance
TBD
Approval Date
Div
id
tabs-2
Software Assurance and Software Safety Requirements Mapping Matrix
Show If
spacePermission
edit
Panel
borderColor
red
titleColor
red
title
Visible to editors only
Requirements content from NPR 7150.2D
Tasking content from NASA-STD-8739.8B
NPR 7150.2 Section
SWE #
NPR 7150.2 Requirement
Software Assurance and Software Safety Tasks
3
Software Management Requirements
3.1
Software Life-Cycle Planning
3.1.2
033
Excerpt Include
SWEHBVD:SWE-033 - Acquisition vs. Development Assessment
SWEHBVD:SWE-033 - Acquisition vs. Development Assessment
nopanel
true
Include Page
SWEHBVD:SWE-033 - Notes
SWEHBVD:SWE-033 - Notes
Include Page
SWE-033 - SA Task1
SWE-033 - SA Task1
Include Page
SWE-033 - SA Task2
SWE-033 - SA Task2
Include Page
SWE-033 - SA Task3
SWE-033 - SA Task3
3.1.3
013
Excerpt Include
SWEHBVD:SWE-013 - Software Plans
SWEHBVD:SWE-013 - Software Plans
nopanel
true
Include Page
SWE-013 - SA Task1
SWE-013 - SA Task1
Include Page
SWE-013 - SA Task2
SWE-013 - SA Task2
3.1.4
024
Excerpt Include
SWEHBVD:SWE-024 - Plan Tracking
SWEHBVD:SWE-024 - Plan Tracking
nopanel
true
Include Page
SWE-024 - SA Task1
SWE-024 - SA Task1
Include Page
SWE-024 - SA Task2
SWE-024 - SA Task2
Include Page
SWE-024 - SA Task3
SWE-024 - SA Task3
3.1.5
034
Excerpt Include
SWEHBVD:SWE-034 - Acceptance Criteria
SWEHBVD:SWE-034 - Acceptance Criteria
nopanel
true
Include Page
SWE-034 - SA Task1
SWE-034 - SA Task1
3.1.6
036
Excerpt Include
SWEHBVD:SWE-036 - Software Process Determination
SWEHBVD:SWE-036 - Software Process Determination
nopanel
true
Include Page
SWE-036 - SA Task1
SWE-036 - SA Task1
Include Page
SWE-036 - SA Task2
SWE-036 - SA Task2
3.1.7
037
Excerpt Include
SWEHBVD:SWE-037 - Software Milestones
SWEHBVD:SWE-037 - Software Milestones
nopanel
true
Include Page
SWE-037 - SA Task1
SWE-037 - SA Task1
Include Page
SWE-037 - SA Task2
SWE-037 - SA Task2
3.1.8
039
Excerpt Include
SWEHBVD:SWE-039 - Software Supplier Insight
SWEHBVD:SWE-039 - Software Supplier Insight
nopanel
true
Include Page
SWE-039 - SA Task1
SWE-039 - SA Task1
Include Page
SWE-039 - SA Task2
SWE-039 - SA Task2
Include Page
SWE-039 - SA Task3
SWE-039 - SA Task3
Include Page
SWE-039 - SA Task4
SWE-039 - SA Task4
Include Page
SWE-039 - SA Task5
SWE-039 - SA Task5
Include Page
SWE-039 - SA Task6
SWE-039 - SA Task6
Include Page
SWE-039 - SA Task7
SWE-039 - SA Task7
Include Page
SWE-039 - SA Task8
SWE-039 - SA Task8
3.1.9
040
Excerpt Include
SWEHBVD:SWE-040 - Access to Software Products
SWEHBVD:SWE-040 - Access to Software Products
nopanel
true
Include Page
SWE-040 - SA Task1
SWE-040 - SA Task1
3.1.10
042
Excerpt Include
SWEHBVD:SWE-042 - Source Code Electronic Access
SWEHBVD:SWE-042 - Source Code Electronic Access
nopanel
true
Include Page
SWE-042 - SA Task1
SWE-042 - SA Task1
3.1.11
139
Excerpt Include
SWEHBVD:SWE-139 - Shall Statements
SWEHBVD:SWE-139 - Shall Statements
nopanel
true
Include Page
SWE-139 - SA Task1
SWE-139 - SA Task1
3.1.12
121
Excerpt Include
SWEHBVD:SWE-121 - Document Tailored Requirements
SWEHBVD:SWE-121 - Document Tailored Requirements
nopanel
true
Include Page
SWE-121 - SA Task1
SWE-121 - SA Task1
Include Page
SWE-121 - SA Task2
SWE-121 - SA Task2
3.1.13
125
Excerpt Include
SWEHBVD:SWE-125 - Requirements Compliance Matrix
SWEHBVD:SWE-125 - Requirements Compliance Matrix
nopanel
true
Include Page
SWE-125 - SA Task1
SWE-125 - SA Task1
Include Page
SWE-125 - SA Task2
SWE-125 - SA Task2
3.1.14
027
Excerpt Include
SWEHBVD:SWE-027 - Use of Commercial, Government, and Legacy Software
SWEHBVD:SWE-027 - Use of Commercial, Government, and Legacy Software
nopanel
true
Include Page
SWE-027 - SA Task1
SWE-027 - SA Task1
3.2
Software Cost Estimation
3.2.1
015
Excerpt Include
SWEHBVD:SWE-015 - Cost Estimation
SWEHBVD:SWE-015 - Cost Estimation
nopanel
true
Include Page
SWE-015 - SA Task1
SWE-015 - SA Task1
3.2.2
151
Excerpt Include
SWEHBVD:SWE-151 - Cost Estimate Conditions
SWEHBVD:SWE-151 - Cost Estimate Conditions
nopanel
true
Include Page
SWE-151 - SA Task1
SWE-151 - SA Task1
3.2.3
174
Excerpt Include
SWEHBVD:SWE-174 - Software Planning Parameters
SWEHBVD:SWE-174 - Software Planning Parameters
nopanel
true
Include Page
SWE-174 - SA Task1
SWE-174 - SA Task1
Include Page
SWE-174 - SA Task2
SWE-174 - SA Task2
3.3
Software Schedules
3.3.1
016
Excerpt Include
SWEHBVD:SWE-016 - Software Schedule
SWEHBVD:SWE-016 - Software Schedule
nopanel
true
Include Page
SWE-016 - SA Task1
SWE-016 - SA Task1
Include Page
SWE-016 - SA Task2
SWE-016 - SA Task2
3.3.2
018
Excerpt Include
SWEHBVD:SWE-018 - Software Activities Review
SWEHBVD:SWE-018 - Software Activities Review
nopanel
true
Include Page
SWE-018 - SA Task1
SWE-018 - SA Task1
Include Page
SWE-018 - SA Task2
SWE-018 - SA Task2
3.3.3
046
Excerpt Include
SWEHBVD:SWE-046 - Supplier Software Schedule
SWEHBVD:SWE-046 - Supplier Software Schedule
nopanel
true
Include Page
SWE-046 - SA Task1
SWE-046 - SA Task1
3.4
Software Training
3.4.1
017
Excerpt Include
SWEHBVD:SWE-017 - Project and Software Training
SWEHBVD:SWE-017 - Project and Software Training
nopanel
true
Include Page
SWE-017 - SA Task1
SWE-017 - SA Task1
Include Page
SWE-017 - SA Task2
SWE-017 - SA Task2
3.5
Software Classification Assessments
3.5.1
020
Excerpt Include
SWEHBVD:SWE-020 - Software Classification
SWEHBVD:SWE-020 - Software Classification
nopanel
true
Include Page
SWE-020 - SA Task1
SWE-020 - SA Task1
3.5.2
176
Excerpt Include
SWEHBVD:SWE-176 - Software Records
SWEHBVD:SWE-176 - Software Records
nopanel
true
Include Page
SWE-176 - SA Task1
SWE-176 - SA Task1
3.6
Software Assurance and Software Independent Verification & Validation
3.6.1
022
Excerpt Include
SWEHBVD:SWE-022 - Software Assurance
SWEHBVD:SWE-022 - Software Assurance
nopanel
true
Include Page
SWE-022 - SA Task1
SWE-022 - SA Task1
3.6.2
141
Excerpt Include
SWEHBVD:SWE-141 - Software Independent Verification and Validation
SWEHBVD:SWE-141 - Software Independent Verification and Validation
nopanel
true
Include Page
SWE-141 - SA Task1
SWE-141 - SA Task1
3.6.3
131
Excerpt Include
SWEHBVD:SWE-131 - Independent Verification and Validation Project Execution Plan
SWEHBVD:SWE-131 - Independent Verification and Validation Project Execution Plan
nopanel
true
Include Page
SWE-131 - SA Task1
SWE-131 - SA Task1
3.6.4
178
Excerpt Include
SWEHBVD:SWE-178 - IV&V Artifacts
SWEHBVD:SWE-178 - IV&V Artifacts
nopanel
true
Include Page
SWE-178 - SA Task1
SWE-178 - SA Task1
3.6.5
179
Excerpt Include
SWEHBVD:SWE-179 - IV&V Submitted Issues and Risks
SWEHBVD:SWE-179 - IV&V Submitted Issues and Risks
nopanel
true
Include Page
SWE-179 - SA Task1
SWE-179 - SA Task1
3.7
Safety-Critical and Mission Critical Software
3.7.1
205
Excerpt Include
SWEHBVD:SWE-205 - Determination of Safety-Critical Software
SWEHBVD:SWE-205 - Determination of Safety-Critical Software
SWEHBVD:SWE-203 - Mandatory Assessments for Non-Conformances
SWEHBVD:SWE-203 - Mandatory Assessments for Non-Conformances
nopanel
true
Include Page
SWE-203 - SA Task1
SWE-203 - SA Task1
Include Page
SWE-203 - SA Task2
SWE-203 - SA Task2
5.5.4
204
Excerpt Include
SWEHBVD:SWE-204 - Process Assessments
SWEHBVD:SWE-204 - Process Assessments
nopanel
true
Include Page
SWE-204 - SA Task1
SWE-204 - SA Task1
Include Page
SWE-204 - SA Task2
SWE-204 - SA Task2
Include Page
SWE-204 - SA Task3
SWE-204 - SA Task3
Include Page
SWE-204 - SA Task4
SWE-204 - SA Task4
Div
id
tabs-3
3.
Example of Table from Software Assurance Plan
Note
The table below was taken from excerpts from Software Assurance Plan in SWEHBVD. The table is built from SWE excerpts plus SA Tasks using the individual SA tasks from the "SA Tasks from NASA-STD-8739.8B" area of SITE.
The advantage of using this technique is that changes to the requirements (from SWEHBVD SWEs) and SA Tasks (from NASA-STD-8739.8B) will be made in one place. Once the updates are made, all of the places where they are repeated (quoted) are automatically updated.
It is a little one time work to setup. It saves time as updates are made in documents.
SWE #
NPR 7150.2 Requirement
NASA-STD-8739.8 Software Assurance and Software Safety Tasks per SA Standard
IV&V is a technical discipline of software assurance that employs rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the software development life The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.) and off-nominal conditions (response to faults, responses to hazardous conditions, etc.). The goal of the IV&V effort is to contribute assurance conclusions provided to the project and stakeholders based on evidence found in software development artifacts and risks associated with the intended behaviors of the software.
Three parameters define the independence of IV&V: technical independence, managerial independence, and financial independence.
Technical independence requires that the personnel performing the IV&V analysis are not involved in the development of the system or its elements. The IV&V team establishes an understanding of the problem and how the system addresses the problem. Through technical independence, the IV&V team’s different perspective allows it to detect subtle errors overlooked by personnel focused on developing the system.
Managerial independence requires that the personnel performing the IV&V analysis are not in the same organization as the development and program management team. Managerial independence also means that the IV&V team makes its own decisions about which segments of the system and its software to analyze and test, chooses the IV&V analysis methods to apply, and defines the IV&V schedule of activities. While independent from the development and program management organization, the IV&V team provides its findings in a timely manner to both of those organizations. The submission of findings to the program management organization should not include any restrictions (e.g., requiring the approval of the development organization) or any other adverse pressures from the development group.
Financial independence requires that the control of the IV&V budget be vested in a group independent of the software development organization. Financial independence does not necessarily mean that the IV&V team controls the budget but that the finances should be structured so that funding is available for the IV&V team to complete its analysis or test work. No adverse financial pressure or influence is applied.
The IV&V process starts early in the software development life cycle, providing feedback to the IV&V provider organization, allowing the IV&V team to modify products at optimal timeframes and in a timely fashion, thereby reducing overall project risk. The feedback also answers project stakeholders’ questions about system properties (correctness, robustness, safety, security, etc.) to make informed decisions with respect to the development and acceptance of the system and its software.
The IV&V provider performs two primary activities, often concurrently: verification and validation. Each of the activities provides a different perspective on the system/software.
Verification is the process of evaluating a system and its software to provide objective evidence as to whether or not a product conforms to the build-to requirements and design specifications. Verification holds from the requirements through the design and code and into testing. Verification demonstrates that the products of a given development phase satisfy the conditions imposed at the start of or during that phase.
Validation develops objective evidence that shows that the content of the engineering artifact is the right content for the developed system/software.
The content is accurate and correct if the objective evidence demonstrates that it satisfies the system requirements (e.g., user needs, stakeholder needs, etc.), fully describes the required capability/functionality needed, and solves the right problem.
The main goal of the IV&V effort is to identify and generate objective evidence that supports the correct operation of the system or refutes the correct operation of the system. The IV&V provider typically works with the development team to understand this objective evidence, which provides artifacts such as concept studies, operations concepts, and requirements that define the overall project. The IV&V provider uses these materials to develop an independent understanding of the project’s commitment to NASA, which forms the basis for validating lower-level technical artifacts.
Two principles help guide the development and use of objective evidence.
Performing IV&V throughout the entire development lifetime is the first principle; potential problems should be detected as early as possible in the development life Performing IV&V throughout the entire development lifetime provides the IV&V team with sufficient information to establish a basis for the analysis results and provides early objective evidence to the development and program management groups to help keep the development effort on track early in the life cycle.
The second principle is “appropriate assurance.” Given that it is not possible to provide IV&V on all aspects of a project’s software, the IV&V provider and project should balance risks against available resources to define an IV&V program for each project that provides IV&V so that the software will operate correctly, safely, reliably, and securely throughout its operational lifetime. The IPEP documents this tailored approach and summarizes the cost/benefit trade-offs made in the scoping process.
The IV&V requirements are analyzed and partitioned according to the type of artifact. The requirements do not imply or require the use of any specific life cycle model. It is also important to understand that IV&V applies to any life cycle development process. The IV&V requirements document the potential scope of analysis performed by the IV&V provider and the key responsibility of the software project to provide the information needed to perform that analysis. Additionally, the risk assessment is used to scope the IV&V analysis to help determine the prioritization of activities and the level of rigor associated with performing those activities. The scoping exercise results are captured in the IV&V Project Execution Plan, as documented below.
3.2 IV&V Requirements
The responsible project manager shall ensure the performance of the IV&V requirements, as defined in section 4.4.2 of this standard. The IV&V requirements in this section of the standard apply to any project required to have IV&V per the criteria defined in the NASA Software Engineering Requirements, NPR 7150.2. The IV&V requirements apply to all IV&V efforts performed on a software development project, as tailored by the IV&V Project Execution Plan. The IV&V requirements also serve as the definition of what NASA considers IV&V. IV&V is a risk mitigation activity, and as such, the application of IV&V analysis and the rigor of that analysis is driven by the IV&V provider’s assessment of software risk.
The IV&V provider shall conduct planning and risk assessments to determine the specific system/software behaviors (including the software components responsible for implementing the behaviors) to be analyzed.
Note
IV&V is a focused activity that prioritizes IV&V analysis to address the highest developmental and operational software risks. IV&V priority is based on the combination of the potential for software impacts on safety and mission success and the probability factors for latent defects. IV&V analysis activities provide coverage with a degree of rigor that reflects the priority level. The initial planning and scoping effort based on the risk assessment define the starting point for the IV&V analysis. During the life cycle of each IV&V project, continuous and iterative feedback, through the execution of analysis, identification of issues and risks, and the collection of deeper mission understanding, allows IV&V projects to “Follow The Risk” and adjust plans. The planning and scoping effort also aid in establishing the initial relationships between the IV&V provider, the Acquirer, and the Provider.
The IV&V provider shall develop and negotiate an IV&V IPEP with the project.
Note
The IPEP documents the activities, methods, level of rigor, environments, tailoring (if any) of the IV&V requirements, and criteria to be used in performing verification and validation of in-scope system/software behaviors (including responsible software components) determined by the planning and scoping effort. A Provider should use a documented analysis approach to track and manage the IV&V effort aligned with ongoing development project efforts. The IPEP documents which software products are subject to which analyses and which analysis requirements are wholly, partially, or not applied following the risk assessment and resource constraints. The IPEP also serves as a communication tool between the project and IV&V to set expectations for the IV&V products produced throughout the life The IPEP may require updating throughout the life cycle.
The Project SMA Technical Authority (TA) shall review and concur with the IPEP.
The IV&V provider shall provide analysis results, risks, and assurance statements and data to the responsible organizations’ project management, engineering, and software assurance personnel.
Note
While independent, the IV&V provider is still a part of a project's overall safety and risk mitigation software assurance strategy. The results of IV&V analysis need to be incorporated into the overall software assurance assessment of the project and provided to the project management. The IV&V provider should support project milestone reviews and provide the project with an evaluation of the life cycle review artifacts to assist development management decisions on whether the review criteria have been met and how to proceed going forward.
The IV&V provider shall participate in project reviews of software activities. Participation includes providing status and results of software IV&V activities including, but not limited to, upcoming analysis activities, artifacts needed from the project, the results of the current or completed analysis, defects, and risks to stakeholders, customers, and development project personnel.
Note
The most significant positive impact of IV&V analysis is when the analysis results are in phase with the development effort. Communicating defects after development artifacts are baselined increases the cost to make the changes. Additionally, the inclusion of the IV&V provider in ongoing technical meetings keeps the IV&V provider informed of possible changes that may affect future IV&V tasking. Supporting the ongoing technical meetings allows the IV&V Provider an opportunity to provide real-time feedback on these changes.
The IV&V provider shall provide the responsible organizations’ project management, engineering, and software assurance personnel insight into the software IV&V and IV&V test activities. As a minimum, the IV&V provider will be required to allow the responsible organizations’ project management, engineering, and software assurance personnel to perform the following activities:
Monitor the IV&V activities and plans.
Review the verification activities to ensure adequacy.
Review IV&V studies and source data.
Audit the software IV&V processes and practices.
Participate in IV&V software reviews and technical interchange meetings
The IV&V provider shall participate in planned software peer reviews or software inspections guided by the planning and scoping risk analysis documented in the IPEP and NASA-HDBK-2203.
Note
The IV&V provider should be involved in the review/inspection process for all system/software items within the scope of their analysis.
4.4.2.8 The IV&V provider shall establish, record, maintain, report, and utilize IV&V management and technical measurements.
Note
The IV&V provider gathers and analyzes metrics on a periodic basis to perform continuous improvement of IV&V processes and identify indicators of IV&V and project risks.
4.4.2.9 The IV&V provider shall assess and track software activities' actual results and performance against the software plans and identify and report any risks or findings to the responsible organizations’ project management, engineering, and software assurance personnel.
4.4.2.10 The IV&V provider shall track and evaluate changes to software products to evaluate for possible changes in the IV&V provider’s risk analysis and potential adverse impacts to the software system and the development effort.
4.4.2.11 The IV&V provider shall assess the software development life cycle for suitability for the problem to be solved and identify and communicate any risks associated with the chosen life cycle to the responsible organizations’ project management, engineering, and software assurance personnel.
4.4.2.12 The IV&V provider shall identify, analyze, track, and record risks to the software and development project in accordance with NPR 8000.4, Agency Risk Management Procedural Requirements, and communicate the risks to the responsible organizations’ project management, engineering, and software assurance personnel.
4.4.2.13 The IV&V provider shall verify the project implements the requirements for software listed in NPR 7150.2 and communicate any risks to the responsible organizations’ project management, engineering, and software assurance personnel.
4.4.2.14 The IV&V provider shall track, record, and communicate defects/issues and other results found during the execution of IV&V analysis and independent IV&V testing to the responsible organizations’ project management, engineering, and software assurance personnel.
4.4.2.15 The IV&V provider shall ensure that the identified defects and issues are addressed by the project.
4.4.2.16 The IV&V provider shall ensure that software planned for reuse meets the fit, form, and function as a component within the new application.
4.4.2.17 The IV&V provider shall ensure that the system architecture contains the computing-related items (subsystems, components, etc.) to carry out the system's mission and satisfy user needs and operational scenarios or use cases.
4.4.2.18 The IV&V provider shall ensure that the basis for the computing-related functions reflects the planned operations and mission objectives.
4.4.2.19 The IV&V provider shall ensure that feasibility and trade studies provide the results to support the critical decisions that drove the need for the study.
4.4.2.20 The IV&V provider shall ensure that known software-based hazard causes, contributors, and controls are identified, documented, and traced to the project requirements.
4.4.2.21 The IV&V provider shall ensure that known security threats and risks are identified, appropriately documented, and updated throughout the software development life cycle and communicated to the responsible organizations’ project management, engineering, and software assurance personnel.
The IV&V provider shall verify and validate
4.4.2.22 That the software requirements and system requirements are, as a minimum, correct, consistent, complete, accurate, readable, traceable, and testable.
Note
Software usually provides the interface between the user and the system hardware and the interface between system hardware components and other systems. These interfaces are critical to the successful operation and use of the system.
4.4.2.23 The IV&V provider shall verify and validate that the mitigations for identified security risks are in the software requirements.
Note
Security is an essential aspect of any system development effort. In most systems, software provides the primary user interface. The user interface is an element of the system that can provide undesired access. A system concept design should address known security risks through various features in the system.
4.4.2.24 The IV&V provider shall ensure that software requirements meet the dependability and fault tolerance required by the system.
4.4.2.25 The IV&V provider shall ensure that software requirements provide the capability of controlling identified hazards and do not create hazardous conditions.
The IV&V provider shall verify and validate
4.4.2.26 that the relationship between the in-scope system/software requirements and the associated architectural elements is traceable, correct, consistent, and complete.
Note
Architectural elements are responsible for implementing specific behaviors within the software and the overall system. The interactions between these architectural elements result in the realization of the desired behaviors as well as possible undesired behaviors.
4.4.2.27 The IV&V provider shall verify and validate that the software architecture meets the user’s safety and mission-critical needs as defined in the requirements.
Note
The architecture provides the foundation for the development of the software. It also significantly impacts how the software deals with faults and failures and how the software interfaces with the user and system components. Analysis of the architecture provides early insight into how the software is structured and how that structure can implement the requirements.
4.4.2.28 The IV&V provider shall verify and validate that the detailed design products are traceable, consistent, complete, accurate, and testable.
Note
Detailed design is the implementation of the algorithms that control and monitor the different parts of the system and allow for interaction between the system and the user and other systems. The detailed design defines how the architectural components behave to support the interactions defined in the architecture. Analysis of the detailed design includes looking at the low-level software components in the software system.
4.4.2.29 The IV&V provider shall verify and validate that the interfaces between the detailed design components and the hardware, users, operators, other software, and external systems are correct, consistent, complete, accurate, and testable.
Note
While the architecture defines the interactions between the architectural elements, each element is generally composed of lower-level components defined by the detailed design. The interfaces between these components are important in ensuring that the architectural element meets its assigned responsibilities.
4.4.2.30 The IV&V provider shall verify and validate that the relationship between the software requirements and the associated detailed design components is correct, consistent, and complete.
Note
The detailed design components capture the approach to implementing the software requirements, including the requirements associated with fault management, security, and safety. Analysis of the relationship between the detailed design and the software requirements provides evidence that all requirements are in the detailed design.
The IV&V provider shall verify and validate
4.4.2.31 That the software code and data products are consistent with architecture, complete with respect to requirements, and testable.
4.4.2.32 The IV&V provider shall verify and validate that the software code meets the industry best practices and software coding standards.
4.4.2.33 The IV&V provider shall verify and validate that the security risks in the software code are identified and mitigated.
Note
Note: This includes software developed by NASA, software developed for NASA, software maintained by or for NASA, COTS, GOTS, MOTS, OSS, reused software components, auto-generated code, embedded software, the software executed on processors embedded in programmable logic devices, legacy, heritage, applications, freeware, shareware, trial or demonstration software.
4.4.2.34 The IV&V provider shall verify and validate the appropriate use of off-the-shelf software, including ensuring that the project has identified all OSS used and that the security risks are identified and mitigated by the use of the off-the-shelf
4.4.2.35 The IV&V provider shall verify and validate that the project assesses the software systems for possible security vulnerabilities and weaknesses.
4.4.2.36 The IV&V provider shall verify and validate that the project implements the required software security risk mitigations to ensure that security objectives for software are satisfied.
4.4.2.37 The IV&V provider shall verify and validate the source code through the use of analysis tools (including but not limited to static, dynamic, composition, and formal analysis tools).
Note
The use of analysis tools may include the verification and validation of the analysis tools used by the development project in the process of developing the software. The results may be from static code analysis, software composition analysis, dynamic code analysis, cyclomatic complexity, or other code quality analysis tools.
4.4.2.38 The IV&V provider shall verify and validate that the relationship between the software design elements and the associated software units is correct, consistent, and complete.
4.4.2.39 The IV&V provider shall verify and validate that the relationship between software code components and corresponding requirements is correct, complete, and consistent.
Note
For all of the implementation requirements, it is with code that the development of software reaches its lowest level of abstraction and that the software capabilities are implemented. Evaluating the relationship between the source code and the design components and requirements provides evidence that only the specified requirements and components are in the system. Evaluating the relationship between the source code and the design components and requirements helps minimize one aspect of the emergence of unexpected behaviors: the inclusion of behaviors not specified in the requirements. The overall analysis of the code is essential in assuring that the code does implement the required software behaviors. From a safety perspective, it is important to evaluate the code and assure that known software safety and security issues such as buffer overflows and type mismatches, among many others, are not used in safety-critical aspects of the software.
4.4.2.40 The IV&V provider shall verify and validate that test plans, test procedures, test cases, test environment (including simulations), and test design at all levels of testing (unit, integration, system, acceptance, etc.) are correct, complete, and consistent for verification and validation of the source code and system functions allocated to the software.
4.4.2.41 The IV&V provider shall verify and validate the relationships between the test plans, test procedures, test cases, test design, source code. and system functions allocated to the software are correct, complete, and consistent.
4.4.2.42 The IV&V provider shall verify that the test plans, test cases, test design, and test procedures contain objective acceptance criteria that support the verification of the associated requirements for both nominal and off-nominal conditions.
4.4.2.43 The IV&V provider shall verify that the software test results meet the associated acceptance criteria to ensure that the software correctly implements the associated requirements.
Note
The IV&V provider assesses the testing artifacts with respect to the desired capabilities and expected operational system environment. The assessment includes an examination of testing at system boundary conditions to include unexpected conditions. The testing analysis assures that the project tests all requirements and that the system does what the requirements state it should do. The testing analysis also includes an analysis of the traceability information between the tests and the requirements.
4.4.2.44 The IV&V provider shall verify that the project tests the required software security risk mitigations to ensure that the security objectives for the software are satisfied.
4.4.2.45 The IV&V provider shall verify that code coverage is measured by analysis of the results of the execution of tests.
4.4.2.46 The IV&V provider shall verify through independent testing each of the software requirements that trace to a hazardous event, cause, or mitigation technique.
4.4.2.47 The IV&V provider shall verify the project’s acceptance tests for loaded or uplinked data, rules, and code that affects software and software system behavior.
4.4.2.48 The IV&V provider shall participate in all NASA quality audits, assessments, and reviews associated with the project.
4.4.2.49 The IV&V provider shall assess the software maintenance and operational risks concerning software elements to support the planning of IV&V activities during the maintenance phase.
Note
The approach to software development on some projects results in different parts of the software going into operation at different times in the overall project life For example, a lander mission to Mars may complete the software needed for the cruise phase to Mars while continuing to work on the entry, descent, landing, and surface operations software.
Note
In some cases, software anomalies cause changes to the software. IV&V is important because software changes can often have ripple effects throughout the system and cause emergent behaviors. The IV&V analysis provides insight into these possible effects and provides an overall assessment of the impact of the change.
Note
This example is taken from SWEHBVD: SWE-013 - Software Plans. It uses the excerpt from tab 1 of the SWE and some include pages for appropriate tasks in the NASA-STD-8739.8B page set in SITE.
