1. Introduction

A primary function of software assurance is to help determine the status of the software project and the quality of the products being produced. The Status Report is a scheduled periodic communication tool to help manage expectations between the SA and Safety Assurance representative(s), project engineering and management, and OSMA stakeholders. It provides insight into the overall status relative to the value SA adds to the project as well as performance against the SA plan. Pre-coordinate with the stakeholders receiving the status reports and define the specifics/content of the status report in the SA Plan. Status reports should provide the information to satisfy the needs of the stakeholders receiving the report. The SA teams will probably be participating in status reporting presented during reviews, such as the milestone reviews and they should also be giving status reports to their management, the project management, and certain other stakeholders more frequently to keep them well informed. When SA has findings or issues of potential high risk, they should not wait until the regularly scheduled status reports to bring those to management.

The information in this topic is divided into several tabs as follows:

• Tab 1 – Introduction
• Tab 2 – Recommended Content – provides guidance on the minimum SA Status report product content (Note: The contents in Tab 2 was previously in 7.18 as SASTATUS.)
• Tab 3 – High Level Summary – provides additional guidance and examples of high-level summary of SA activities that may be included in the status report when the reporting time slot is very short.
• Tab 4 – SA Analysis – provides reporting guidance on analyses performed during a reporting period
• Tab 5 – SA Assessments – provides guidance on assessments performed and reporting on them
• Tab 6 – SA Audits – provides reporting guidance on audits performed during a reporting period
• Tab 7 – Resources for this topic

The following is a list of the applicable SWE requirements that relate to the generation of SA Status Reports:

1.1 Additional Guidance

Links to Additional Guidance materials for this subject have been compiled in the Relevant Links table. Click here to see the  in the Resources tab.

2. Recommended Content

2.2 Additional Guidance

Links to Additional Guidance materials for this subject have been compiled in the Relevant Links table. Click here to see the  in the Resources tab.

3. High Level Summary

In cases where time is critical, it may be necessary to provide management and stakeholders with a high-level SA Status Summary Report. This report should include a brief summary of the activities performed by the Software Assurance or Software Safety personnel during the reporting period to give an overall project status at a glance. (Note: This is not intended to be a substitute for a full status report as described on Tab 2. Recommended Content.)

The specific content of a high-level SA Status Summary Report should be pre-coordinated and agreed to by the customers of the Status Report. Although this Handbook provides a recommended set of content (see below), projects have the liberty to adjust it to meet their specific needs. Any high-risk items or issues should always be included in the status reports as well as any key information from any of the listed analyses that have not been reported on previously or that have provided new information.

When preparing a high-level SA Status Summary Report, the following defines the minimum recommended contents:

• High-level status of progress and schedule. Include:
  • Overall evaluation, based on judgement
  • Issues and concerns,
  • Whether or not assistance/awareness is required.
• High-level summary of work performed – Identify what was done, overall evaluation, and number of findings for:
  • Assessments, analyses, and audits performed
  • Products reviewed, and
  • Number of Tests Witnessed
• High-level summary of work in-progress
• High-level summary of upcoming work
• Major findings and associated risk – These could be newly identified or existing.
• Risk summary – These could be newly identified or high-visibility/high-priority risks.
• Metrics – Current status of findings/corrective actions: open/closed; projection for closure timeframe. For example, a trend chart (e.g., testing progress, non-conformance closure rate), or product quality attributes (# of SA findings per review or audit).

This information could be presented in many ways such as:

• Dashboard or Stoplight Table – Typically, a Green/Yellow/Red scheme is used for indicating go, no-go status and approaching minimum threshold or limits.
• Quad Chart – A chart with 4 sections e.g., Project Status, Work Completed, Upcoming/In-Progress Work, and Risks/Issues/Concerns
• Trend Chart – A chart that shows changes of a metric over time, such as requirements volatility over time or closure of discrepancies over time.
• Burndown Chart – A chart to show project progress (i.e., work completed and work remaining) for a certain time frame. This could be for an entire project or for an Agile sprint.

If major findings or issues are high risk items, they should be reported to management and the development team immediately. Project managers and software leads don’t like to be surprised, so the sooner the issue/risk becomes known, the sooner it can be addressed and the better the chance that the impact can be minimized.

3.1 Additional Guidance

Links to Additional Guidance materials for this subject have been compiled in the Relevant Links table. Click here to see the  in the Resources tab.

See also SWE-018 - Software Activities Review, 

4. SA Analysis

What does it mean to perform an “analysis”? The Software Assurance and Software Safety Standard (SASS), NASA-STD-8739.8, defines “analyze” as:

Review results in-depth, look at relationships of activities, examine methodologies in detail, follow methodologies such as Failure Mode and Effects Analysis, Fault Tree Analysis, trending, and metrics analysis. Examine processes, plans, products, and task lists for completeness, consistency, accuracy, reasonableness, and compliance with requirements. The analysis may include identifying missing, incomplete, or inaccurate products, relationships, deliverables, activities, required actions, etc.

Thus, the analyses performed by Software Assurance and Software Safety personnel are intended to provide an in-depth look at the products generated by the software engineering organization during the software development life cycle. These analyses are more than reviews as they require personnel to study and scrutinize the engineering products to identify strengths and weaknesses (e.g., in the design, code, test coverage, security vulnerabilities, hazard analysis) as well as errors, potential problems, issues and risks. Each required analysis type is covered in detail in other sections of Topic 8.16 – SA Products and are listed below:

• 8.54 - Software Requirements Analysis
• 8.55 - Software Design Analysis
• 8.56 - Source Code Quality Analysis
• 8.57 - Testing Analysis
• 8.58 - Software Safety and Hazard Analysis

There are two additional analyses not covered under separate topics that Software Assurance performs. They are:

• Verification Activities Analysis – Software Verification is the “confirmation that products properly reflect the requirements specified for them” – NASA-STD-8739.8. As such, any analysis performed on verification activities should be included in status reporting, particularly if any findings show problems that may be difficult/time-consuming to fix or cause a redesign/or considerable rework.
• Root Cause Analysis – If any root cause analysis has been performed on a high severity non-conformance, report the results as well as the impact on the project, plans to identify similar problems in other areas of the project, and the selected method of preventing similar problems in future projects.

During each status reporting period, a high-level summary of any analysis done during the reporting period should be reported. The specific SA Status Report content for each SA product listed above is defined in the product pages. However, in general, when reporting the results of an analysis in a SA Status Report, the following defines the minimum recommended contents:

• Identification of what was analyzed: Mission/Project/Application
• Period/Timeframe/Phase analysis performed during
• Summary of analysis techniques used
• Overall assessment of the software engineering work product, based on analysis
• Major findings and associated risk
• Current status of findings: open/closed; projection for closure timeframe

If a time-critical issue is uncovered, it should be reported to management immediately so that the affected organization may begin addressing it at once.

4.1 Additional Guidance

Links to Additional Guidance materials for this subject have been compiled in the Relevant Links table. Click here to see the  in the Resources tab.

5. SA Assessments

What does it mean to perform an “assessment”? The Software Assurance and Software Safety Standard (SASS), NASA-STD-8739.8 , defines “assess” as:

Judge results against plans or work product requirements. “Assess” includes judging for practicality, timeliness, correctness, completeness, compliance, evaluation of rationale, etc., reviewing activities performed, and independently tracking corrective actions to closure.

Therefore, when performing an assessment/evaluation of a project product or doing a compliance check, SA and Software Safety personnel will use their judgement to determine if the item being evaluated meets expectations.

There are 29 SASS tasks in NASA-STD-8739.8 that require the Software Assurance or Software Safety personnel to “assess” something and another set of tasks where they are required to confirm that an assessment has been done.

The number of assessments required may vary based on the software classification and the approved tailoring of the SWE and SASS requirements in the Requirements Mapping Matrix. When appropriate, each assessment should be performed in the timeframe corresponding to the performance of the associated SWE task. See the table below for the list of required assessments in NASA-STD-8739.8.

The results of any assessments performed are recorded and reported on at either (or both) the next regular management meeting or included in the SA Status Report. Depending on the type of assessment performed, it may be more appropriate to document the results in the associated SA analysis product rather than a status report. For example, an assessment performed on the software architecture (SWE-057) should be documented as part of the 8.55 - Software Design Analysis product as it is relevant to the overall analysis.  If there is no related SA product, document it separately and include the results in the SA Status Report for the applicable reporting period.

When reporting the results of an assessment in a SA Status Report, the following defines the minimum recommended contents:

• Identification of what was assessed: Mission/Project/Application
• Type of Assessment
• Period/Timeframe/Phase assessment performed during
• Overall evaluation of the assessment target
• Major findings and associated risk
• Current status of findings: open/closed; projection for closure timeframe

Note: Table below was done with NASA-STD-8739.8B draft as of 11/10/21.Will need updating when Rev B is official.

List of Assessments in NASA-STD-8739.8

5.1 Additional Guidance

Links to Additional Guidance materials for this subject have been compiled in the Relevant Links table. Click here to see the  in the Resources tab.

6. SA Audits

The Software Assurance and Software Safety Standard (SASS), NASA-STD-8739.8 , defines “audit” as:

• systematic, independent, and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which audit criteria are fulfilled
• independent examination of a work product or set of work products to assess compliance with specifications, standards, contractual agreements, or other criteria
• independent examination of a software product, software process, or set of software processes to assess compliance with specifications, standards, contractual agreements, or other criteria

• systematic, independent, documented process for obtaining records, statements of fact, or other relevant information and assessing them objectively, to determine the extent to which specified requirements are fulfilled. Note: An audit can be an internal audit (first-party) or an external audit (second party or a third party), and it can be a combined or integrated audit (combining two or more disciplines). Audit results are a clear indication of whether the audit criteria have been met. (IEEE Definition

Therefore, when performing audits, SA personnel are checking to determine compliance or the extent to which certain requirements are fulfilled. SA may also use audits to assess work being performed using an established set of expectations (e.g., coding standards, software engineering work product content – see 7.18 - Documentation Guidance).

There are 9 SASS tasks in NASA-STD-8739.8 that require the Software Assurance or Software Safety personnel to “audit” processes, procedures, and standards. (See Topic 8.16 – Audit Reports (Tab 2) for the list of audits). Due to the nature of some SASS audit requirements, the findings from audits may provide information on process-related problems rather than specific technical issues. Both types are important to bring to management’s attention early so they can be addressed as soon as possible.

The results of any audits performed are recorded, and reported on at either (or both) the next regular management meeting or included in the SA Status Report. There are two types of Audit Reports – Audit Summary and Detailed Audit Report. For SA Status Reports, the Audit Summary content should be used.

See Topic 8.16 - Audit Reports (Tab 4) for report contents.

For more information on conducting audits, see Topic 8.12 – Basics of Software Auditing.

6.1 Additional Guidance

Links to Additional Guidance materials for this subject have been compiled in the Relevant Links table. Click here to see the  in the Resources tab.

7. Resources

7.1 References

Enter necessary modifications to be made in the table below: SWEREFs to be added SWEREFS to be deleted SWEREFs called out in text: 278 SWEREFs NOT called out in text but listed as germane: 048, 083, Related Links Pages Refstable Topic

7.2 Tools

7.3 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

7.4 Center Process Asset Libraries

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only).