1. Engineering Questions

2. Software Assurance Questions

 1. Please explain how the requirements in NASA-STD-8739.8 can be tailored.

There are three levels of tailoring applicable for the software assurance and safety requirements in NASA-STD-8739.8. They are as follows:

1. The first level of tailoring is the Software Classification Decision, see NPR 7150.2. See Diagram 1 from the NPR.
2. The second level of tailoring is the tailoring in the project’s Software Requirements Mapping Matrix, see NPR 7150.2. See Diagram 2 showing a sample Software Requirements Mapping Matrix.
3. The third level of tailoring is the tailoring by the SA TA of the requirements generated by the corresponding SA requirements to the project’s Software Requirements Mapping Matrix requirements.

Diagram 1

Diagram 2

2. Can programs choose to “opt-out” of this Standard, like engineering may be staying on 7150.2B? That is, are 7150.2C/8739.8A intimately linked so you must move to them together?

It is highly desirable for the programs/projects still using the B version of the NPR to move to the latest version of the Standard, NASA-STD-8739.8A.

3. Can engineering move to the new revision and SMA keep the old and vice versa?

It is possible for engineering to use NPR 7150.2B 

4. In the old standard, sections 5.5.1.4, 6.6.2 and 6.7.1 referenced the building of “lessons learned” throughout a project.   In the new Standard, there is no reference to lessons learned.   What is the reasoning behind this exclusion?

• “5. Responsibility:
  …..
  b. The NASA CKO is responsible for policy and integration of knowledge services across programs and projects in the Centers and Mission Directorates and reports to the Office of the Chief Engineer. The NASA CKO shall:
  (8) Facilitate the dissemination and promote the infusion and utilization of LESSONS LEARNED and best practices, develop and share actionable methodologies, and maintain the NASA LESSONS LEARNED Information System.”

5. What value does test witnessing provide?

The person(s) performing test witnessing should be able to:

1. Confirm that the test environment is set up and configured properly,
2. Confirm that each test case is run correctly, according to the agreed upon test procedures,
3. Confirm that the test produces the expected results, and
4. Confirm that any defects, discrepancies, or problems with the test environment, test configuration, test procedures/cases, test run, or results are documented and addressed before the test is considered complete, either with a “passing result” or in accordance with the acceptance criteria. Ideally:
   1. All tests should result in a “pass”.
   2. If there are defects, discrepancies, or problems preventing a passing result, confirm that they are discussed with or evaluated by the Customer to determine if it must be fixed or if the software can be released with the bug. This will allow the test to be closed out.
   3. Note: If there is a problem with the test environment or test configuration, the test should be cancelled.

6. What differences are being introduced with the updated SA Standard (NASA-STD-8739.8)? 

The three main differences are the inclusion of the secure coding practices,  the cybersecurity requirements and revised criteria for determining safety-critical software. This revision of the SA Standard is closely tied to the NPR 7150.2, so it is obvious what assurance and safety practices need to be performed for each NPR requirement. The SW Safety Standard (NASA-STD-8719.13) has also been rolled into the new SA Standard. The new Standard is much more performance based.

3. Safety FAQ

1. What should we be doing to identify what software might be safety critical under the new Standard (NASA-STD-8739.8A

Swerefn
refnum 278

)?

NASA-STD-8739.8A defines a safety criticality software determination process which should be used to identify safety critical software:

Software is classified as safety-critical, if it meets at least one of the following criteria:

• • Causes or contributes to a system hazardous condition/event,
  • Provides control or mitigation for a system hazardous condition/event,
  • Controls safety-critical functions,
  • Mitigates damage if a hazardous condition/event occurs,
  • Detects, reports, and takes corrective action, if the system reaches a potentially hazardous state.

Look at the systems hazard analysis and see if there is any software that meets any of the above criteria. If so, it is safety critical

2. If the software could affect a command, but there are features built in to ensure the problem will not occur, is it safety critical?

No.

3. What additional requirements or activities are done for safety critical software?

The majority of the activities for safety critical software are just good engineering practices. The new Standard only has two new requirements for safety critical software that were not in the old Standard:

1. Achieve 100% code coverage when testing safety critical software. (If it can’t be achieved, a rational for why it can’t be reached must be provided)
2. The Cyclomatic Complexity of safety critical software must be 15 or lower.

Other important activities for safety critical software include:

• • Perform an assessment of the hazard analysis to identify software components associated with system hazards as per the safety criticality determination listed of Question 1.
  •  Develop and maintain a safety analysis throughout the life cycle.
  • Assess that the code meets the requirements in SWE-134.
  • Perform a software assurance design analysis.
  • Confirm that the software test plan addresses the verification of safety-critical software, specifically the off-nominal scenarios.
  • All safety critical software needs to be verified by test. All critical lines of code must be tested as well as all conditions and end points
  • Confirm that the traceability between software requirements and hazards with software contributions exist.
  • Perform a software assurance analysis on the detailed software requirements to analyze the software requirement sources and identify any incorrect, missing, or incomplete requirements.
  • Confirm that the software design implements all of the required safety-critical functions and requirements.
  • Perform test witnessing for safety-criticality software.
  • Adequate regression testing is performed and includes retesting of all safety-critical code components.

It is important to make sure you have a good hazard analysis. It tells what requirements are safety critical to indicate which elements of the design, code, and test activities need additional rigor.

4. Do we still need to mark the safety critical code?

Marking (“tagging”) of the software is no longer required. 

5. Do we still figure out the “levels” of safety critical software (i.e., use of the levels to scope the work on the safety critical software) like it is described in the Software Safety Guidebook?

The sub-levels are not needed any more---either the software is safety critical or it is not.

6. The previous method called for a formal preliminary hazard analysis, plus a hazard analysis, now what do we need? Do we need explicit documentation on safety critical software? When would you see it?

The hardware and software groups should cooperate on developing the hazard analysis reports. You need to have a plan on how you plan to handle the safety critical software. Now, with the new approach, software by itself must cause a hazard for it to be safety critical. This may be mitigated with inhibits or just declare it safety critical. The new approach just requires a few new steps and tracing to hazards. 

7. What about mission critical software?

Still need to consider SWE-134 for mission critical software.

8. Is verification by similarity allowed for safety critical software?

No, safety critical software is required to be verified by test. See the requirements below.

NPR 7150.2C

Swerefn
refnum 083

4.5.12 The project manager shall verify through test the software requirements that trace to a hazardous event, cause, or mitigation technique. [SWE-192]

NASA-STD-8739.8A 

Swerefn
refnum 278

A project can tailor the requirement if the engineering and SMA TAs agree with the rational and risk.

9. In the new NASA-STD-8739.8, safety and software assurance are combined, but they are often performed by different roles. Will the training for each be combined or will there be separate training for each role?

The SMA Technical Training Program (STEP) program has separate paths of study for the software assurance and software safety roles. However, the training classes for each are available individually. The classes are available in SATERN to anyone who wants to take them, so the individual may select and take a designated class if desired. The STEP program is a tiered, coordinated set of classes and on-the-job-training to prepare participants for certain roles.

4. Resources

4.1 References