6. Lessons Learned

6.1 NASA Lessons Learned

No Lessons Learned have currently been identified for this requirement.

6.2 Other Lessons Learned

No other Lessons Learned have currently been identified for this requirement.

7. Software Assurance

7.1 Tasking for Software Assurance

7.2 Software Assurance Products

• Software Assurance and Safety Requirements Mapping Matrix  See 8.15 - SA Tasking Checklist Tool for guidance on Software Assurance and Safety Requirements Mapping Matrix 
• Software Assurance Plan (see topic 8.16 - 8.51 - Software Assurance Plan defined for software assurance, including safety, if required)
• Software Safety Plan (see Software Safety Plan tab located under topic 8.16 - 8.51 - Software Assurance Plan)

7.3 Metrics

•  # of software work product Non-Conformances identified by life cycle phase over time·
• Identify the specific requirements in NASA-STD-8739.8 that are being tailored by the projects (*organizational metric)
• # of Non-Conformances identified in plans (e.g., SMPs, SDPs, CM Plans, SA Plans, Safety Plans, Test Plans)
•  # of projects tailoring each requirement (*organizational measure)
•  % of requirements tailored per project (*organizational measure)
• # of safety-related non-conformances identified by life cycle phase over time

7.4 Guidance

Step 1 Confirm that all appropriate plans are in place, and have expected content for the life cycle events, with proper tailoring for the classification of the software.

Software plans include, but are not limited to:

1. Software development or management plan.
2. Software configuration management plan.
3. Software test plans.
4. Software maintenance plans.
5. Software assurance plans.
6. The software safety plan, if the project has safety-critical software.
7. IT Security plans

When software assurance is confirming the appropriate software plans are in place, the following plans should be considered: SW Management/Development Plans, SW Configuration Management Plans, SW Risk Management Plan, SW V&V Plans, SW Operations/Maintenance Plan(s), SA Plans, SW Retirement Plan, and SW Safety Plan(s), if applicable. As mentioned earlier, these plans do not need to all be separate plans, but the applicable contents should be covered among the project’s plans. Recommended minimum contents for these plans can be found in NASA-HDBK-2203, under 7.18 - Documentation Guidance.

Some of these documents may not be available early in the life cycle. Topic 7.8 08 - Maturity of Life Cycle Products at Milestone Reviews will provide some guidance on when these documents should be available and baselined.

Step 2 Develop a Software Assurance Plan following the content defined in NASA-HDBK-2203 for a software assurance plan, including software safety if required.

Software assurance plans should be based on the software assurance tasking identified in the Software Assurance Tasking Checklist Tool. See 8.15 - SA Tasking Checklist Tool

The purpose of the Software Assurance (SA) Tasking Checklist Tool is to streamline the Software Assurance and Software Safety Tasks (a.k.a. SA Tasking) that must be performed on a NASA project. The tool allows the user to tailor the SA Tasking based on the needs of the project via Software Classification, Safety Criticality, and required Milestones.

The SA Tasking Checklist Tool assists in the planning, execution, and monitoring of the Software Assurance (SA) tasks provided in NASA-STD-8739.8A Requirement SASS-01, mapped to the NPR-7150.2C Software Engineering Requirements. The tool is designed with a user-friendly front end which integrates the engineering, software assurance, and safety requirements across the development life cycle to create SA tasking checklists based on milestones to plan and ensure compliance. While the default project information addresses a “typical” development project with full compliance to SASS-01, the tool is flexible in terms of tailoring the requirements, as well as providing the ability to map the SWE requirements to various milestones for different development life cycles to address Center or project-specific attributes. The tool may also be used to capture status when SA activities are performed throughout the development life cycle. The resulting checklist of SA Tasking may be filtered. Monitoring of the resulting SA Tasking may be performed using the generated checklist in the tool. Another option for monitoring is to export the checklist(s) in common formats compatible with other tools including Excel, JIRA, and MS Project (i.e., Excel, CSV, and XML).

The SA Tasking Checklist Tool has a comprehensive Users Guide embedded in the tool to assist the user with tool features and functionality. It also provides instruction on how to use the tool to generate a project-specific SA Tasking Checklist.

For Software Assurance Plan content guidance see 8.51 - Software Assurance Plan.  The major part of the software assurance plan should be the software assurance and software safety requirements mapping matrix. An output from the 8.15 - SA Tasking Checklist Tool can be used for the software assurance and software safety requirements mapping matrix.

The software assurance plan can be a document or an electronic record. 

The SA plan should include a record for the Software Assurance and Software Safety Standard requirements tailoring based on the tailoring of the software assurance, software safety, and IV&V requirements using the following levels:

1. 1. The first level of tailoring is the Software Classification Decision, see NPR 7150.2.
   2. The second level of tailoring is the tailoring in the project’s Software Requirements Mapping Matrix, see NPR 7150.2.
   3. The third level of tailoring is the tailoring by the Software Assurance TA of the Software Assurance and Software Safety requirements that correspond to the project’s Software Requirements Mapping Matrix requirements.

Planning the software assurance on any project requires an understanding of the project’s function, needs, and risk posture as well as Software Class and criticality.   Software Assurance and Software Safety needs to work with the software development team to assure that the software development processes are planned well and are based on the project and software criteria, as appropriate. Software Assurance also needs to assure any acquisition process is adequate and complete and that criteria are set up to eventually assure the delivery of needed software products and reports.  Once these criteria are established, software assurance can create their plans and assure the appropriate SA needs are provided, any needed training on the project development processes and functionality, needed tools, and access to project data, products, and activities.

Office of Safety and Mission Assurance is responsible for creating, maintaining, and assuring the proper execution of the basic SA requirements; however, at the Centers and on Projects and Programs, various personnel can perform varying aspects of SA. Based on SMA resources and Project needs and resources, a combination of Project, Project independent SMA personnel, DCMA, support contractors, NASA, and contractor personnel all may serve in some capacity to meet the total SA requirements. All planned SA activities and a listing of the parties responsible for performing these activities need to be documented and follow the organizational structure and governing documents for each Program/Project. The SA personnel has the responsibility to identify SA issues and risks for the Projects they support and to elevate those that remain unresolved through the SMA reporting chain. The responsible SMA organization assures the personnel performing SA are trained per requirements to perform SA and provides them with a reporting chain independent from the organization whose products and processes they are assuring (an independent reporting chain). SA is performed by both the Acquirer (NASA) and the Provider (NASA and/or contractor).

Many different groups may perform the varying aspects of SA (e.g., systems engineering might perform the software safety analyses, software engineering might collect and trend defects). An entity/organization independent from the development organization has to either perform the SA activities or assure that those activities are performed by the development organization correctly and to the extent consistent with the software classification and safety criticality of the software and that records of those activities are created, analyzed, and maintained.  Many software assurance and software safety activities may be tailored and performed within the Project structure, but a group independent from the Project assures those activities and the results.

Often, one or more software assurance personnel from an SMA organization may be assigned to work with a project throughout its life cycle. While these SAs are a part of the Project and participate in day-to-day activities, performing most or all of the SA functions and attending Project meetings and reviews, they maintain a separate reporting chain through their SMA organization. This activity is an oversight role, the SAs are closely tied in with the Project and provide input daily. At other times, the independent SMA organization may provide only insight into the Project, assuring the SA activities are performed by the Project personnel and participating primarily in audits and at formal review intervals. In either case, there has to be a close working association and joint reporting to both the Project and the SMA organization.

Step 3 Assess plans for risks, issues, completeness, and compliance with NPR 7150.2 requirements, NASA-STD-8739.8, contract, and Center requirements.

When software assurance is confirming the appropriate software plans are in place, the following plans should be considered: SW Management/Development Plans, SW Configuration Management Plans, SW Risk Management Plan, SW V&V Plans, SW Operations/Maintenance Plan(s), SA Plans, SW Retirement Plan, and SW Safety Plan(s), if applicable. As mentioned earlier, these plans do not need to all be separate plans, but the applicable contents should be covered among the project’s plans. Recommended minimum contents for these plans can be found in NASA-HDBK-2203, under 7.18 - Documentation Guidance.

Some of these documents may not be available early in the life cycle. Topic 7.8 08 - Maturity of Life Cycle Products at Milestone Reviews will provide some guidance on when these documents should be available and baselined.

In addition to assessing that these documents have their expected contents, they should be evaluated to verify that they:

• satisfy their requirements, per the appropriate software classification (i.e., NPR 7150.2, Center, Project, any contractual requirements, and Project planning requirements, as applicable)
• are consistent among SW plans, including risk management, configuration management, and problem/discrepancy reporting and resolution,
• cover the appropriate topics, and include all life cycle phases,
• are complete and usable,
• are consistent with systems plans, planned communications, deliverables, and schedules, 
• have identified critical paths and dependencies,
• identify risks associated with proposed plans,
• have determined initial safety criticality, potential security risks, coordination with IV&V, as needed
• have complete coverage, with no omissions or inadequacies. 

Software assurance will identify any risks or issues found during their assessment of the plans and report them to software management. Issues and risks found should be tracked to closure.

7.5 Additional Guidance

Additional guidance related to maintaining and executing software plans and processes this requirement may be found in the following requirements materials in this Handbook: