bannerd

A. Introduction

B. Institutional Requirements

C. Project Software Requirements

D. Topics

E. Tools, References, and Terms

F. SPAN
(NASA Only)

Versions Compared

Old Version 4

changes.mady.by.user Haigh, Fred Douglas. (HQ-KA000)[PEROT]

Saved on

compared with

New Version Current

changes.mady.by.user Haigh, Fred Douglas. (HQ-KA000)[PEROT]

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This is the table from Topic 8.18, reformatted a little but without TABSETUP and other things that might break sticky headers. It doesn't have sticky headers and I don't know why. 

Metrics Type

Measurements

PLANREQDESIMPTESTDELAssociated SWE Reqt #

Peer Review Metrics

# of peer reviews performed vs. # of peer reviews planned


X

X

X

X

X

SWE-016

SWE-087

SWE-089

Peer Review Metrics

# of Non-Conformances identified in each peer review


X

X

X

X

X

SWE-087

SWE-089

Peer Review Metrics

# of Non-Conformances identified by software assurance during each peer review


X

X

X

X

X

SWE-087

SWE-088

SWE-089

Peer Review Metrics

Total # of peer review Non-Conformances (Open, Closed)


X

X

X

X

X

SWE-087

SWE-088

SWE-089

Peer Review Metrics

Preparation time each review participant spent preparing for the review


X

X

X

X

X

SWE-088

SWE-089

Peer Review Metrics

Time required to close review Non-Conformances


X

X

X

X

X

SWE-087

SWE-088

SWE-089

Peer Review Metrics

# of peer review participants vs. total # invited


X

X

X

X

X

SWE-088

SWE-089

Peer Review Metrics

# of peer review Non-Conformances per work product vs. # of peer reviewers


X

X

X

X

X

SWE-088

SWE-089

Peer Review Audit Metrics

# of audit Non-Conformances per peer review audit


X

X

X

X

X

SWE-088

Peer Review Audit Metrics

# of Peer Review Audits planned vs. # of Peer Review Audits performed


X

X

X

X

X

SWE-016

SWE-087

SWE-088

Peer Review Audit Metrics

Trends on non-conformances from audits (Open, Closed, Life cycle Phase)


X

X

X

X

X

SWE-088

SWE-089

Peer Review Audit Metrics

Time required to close peer review audit Non-Conformances


X

X

X

X

X

SWE-088

SWE-089

Peer Review Audit Metrics

Preparation time each audit participant spent preparing for audit


X

X

X

X

X

SWE-088

SWE-089

Problem/Change Report Status Metrics

Total # of Non-Conformances over time (Open, Closed, # of days Open, and Severity of Open)

# of Non-Conformances in current reporting period (Open, Closed, Severity)

X

X

X

X

X

X

SWE-062

SWE-065c

SWE-065d

SWE-068

SWE-202

SWE-203

SWE-204

Problem/Change Report Status Metrics

# of safety-related Non-Conformances


X

X

X

X

X

SWE-203

SWE-068

SWE-071

Problem/Change Report Status Metrics

Trend of Open vs. Closed Non-Conformances over time

X

X

X

X

X

X

SWE-053

SWE-054

SWE-065

Problem/Change Report Status Metrics

Trend of change status over time (# of changes approved, # in implementation, # in test, # closed)

X

X

X

X

X

X

SWE-018

SWE-053

SWE-080

Problem/Change Report Status Metrics

# of Non-Conformances identified in embedded COTS, GOT, MOTS, OSS, or reused components in ground or flight software vs. # of Non-Conformances closed



X

X

X


SWE-136

SWE-202

SWE-203

SWE-211

Problem/Change Report Status Metrics

# of Non-Conformances identified in source code products used (Open, Closed)



X

X

X


SWE-202

SWE-203

Problem/Change Report Status Metrics

# of software Non-Conformances at each Severity level for each software configuration item (Open, Closed)

        Bold type indicates a required item

 

 

 

X

X

X

SWE-202

Problem/Change Report Status Metrics

# of Closed action items vs. # of Open action items

X

X

X

X

X

X

SWE-062 SWE-62c

SWE-065d

Problem/Change Report Status Metrics

# of Root Cause Analyses performed;

# of Non-Conformances identified by each root cause analysis



X

X

X

X

SWE-204

SA Corrective Action (CA) metrics (Issues, Risks)

# of Corrective Actions (CAs) raised by SA vs. total #

       Attributes (Type, Severity, # of days Open, Life cycle Phase Found)

      State (Open, In work, Closed)

      Trends of CA Open vs. Closures over time

 X

X

X

X

X

X

SWE-024

SWE-204

SWE-054

SA Corrective Action (CA) metrics (Issues, Risks)

Trend the # of inconsistencies or corrective actions identified, and # closed.

X

X

X

X

X


SWE-054

SWE-024

SWE-204

SA Corrective Action (CA) metrics (Issues, Risks)# of open vs. closed issues over time and latencyXXXXXXSWE-018

Process Improvement Metrics

# of software work product Non-Conformances identified by life cycle phase over time

X

X

X

X

X

X

SWE-013

SWE-022

SWE-024

SWE-039

SWE-051

SWE-054

SWE-057

SWE-058

SWE-062

SWE-065b

SWE-065c

SWE-065d

SWE-068

SWE-071

SWE-075

SWE-079

SWE-084

SWE-086

SWE-087

SWE-125

SWE-134

SWE-139

SWE-146

SWE-157

SWE-159

SWE-184

SWE-185

SWE-187

SWE-191

SWE-194

SWE-201

SWE-204

SWE-205

Process Improvement Metrics

# of software process Non-Conformances by life cycle phase over time

X

X

X

X

X

X

SWE-032

SWE-039

SWE-061

SWE-077

SWE-080

SWE-082

SWE-085

SWE-086

SWE-088

SWE-139

SWE-195

SWE-204

Process Improvement Metrics

Identify the specific requirements in NASA-STD-8739.8 that are being tailored by the projects (*organizational metric)

X

X

X

X

X

X

SWE-121

SWE-125

SWE-013

SWE-176

Process Improvement Metrics

# of projects tailoring each requirement (*organizational measure)

% of requirements tailored per project (*organizational measure)

X

X

X

X



SWE-125

SWE-013

SWE-121

Process Improvement Metrics

% of Total Source Code for each Software Classification (*organizational measure)




X

X

X

SWE-020

SWE-176

SWE-087

Cost/ Effort Metrics

Planned SA resource allocation vs. actual SA resource allocation 

X

X

X

X

X

X

SWE-015

SWE-174

SWE-151

Cost/ Effort Metrics

Comparison of initial SA cost estimates vs. final cost (capturing assumptions and differences)

X





X

SWE-174

SWE-015

SWE-151

Cost/ Effort Metrics

Trend SA cost estimates throughout life cycle

X

X

X

X

X

X

SWE-174

Training Metrics

% of required training completed for each of the project SA personnel

X

X

X

X



SWE-017

Training Metrics

% of project personnel that have completed project specific training against the planned training schedule


X

X

X

X

X

SWE-017

Compliance Audit Metrics

# of Compliance Audits planned vs. # of Compliance Audits performed

X

X

X

X

X

X

SWE-024

SWE-039

SWE-139

SWE-016

SWE-032

SWE-077

SWE-195

SWE-082

SWE-084

SWE-085

SWE-086

SWE-088

SWE-079 SWE-201

Compliance Audit Metrics

# of Open vs. Closed Audit Non-Conformances over time

# of Non-Conformances per audit (including findings from process and compliance audits, process maturity)


X

X

X

X

X

X

SWE-022

SWE-024

SWE-039

SWE-139

SWE-016

SWE-032

SWE-077

SWE-079

SWE-195

SWE-082

SWE-084

SWE-085

SWE-086

SWE-088

SWE-201

Compliance Audit Metrics

# of Non-Conformances identified in plans (e.g., SMPs, SDPs, CM Plans, SA Plans, Safety Plans, Test Plans)

X

X

X

X

X


SWE-024

SWE-013

SWE-075

SWE-079

SWE-139

SWE-071

Compliance Audit Metrics

#  of Non-Conformances identified in the software Configuration Management Plan

Trends of # Open vs. # Closed over time


X

X

X

X

X

SWE-079

Compliance Audit Metrics

#  of Configuration Management Audits conducted by the project – Planned vs. Actual


X

X

X

X

X

SWE-082

SWE-077

SWE-084

Compliance Audit Metrics

# of Non-Conformances per audit (including findings from process and compliance audits, process maturity)

X

X

X

X

X

X

SWE-024

SWE-039

SWE-139

SWE-016

SWE-032

SWE-077

SWE-195

SWE-082

SWE-084

SWE-085

SWE-086

SWE-088

SWE-079

Compliance Audit Metrics

# of process Non-Conformances (e.g., activities not performed) identified by SA vs. # accepted by the project

Trends of # Open vs. # Closed over time

X

X

X

X

X

X

SWE-016

SWE-039

SWE-032

SWE-077

SWE-195

SWE-082

SWE-084

SWE-085

SWE-086

SWE-088

Compliance Audit Metrics

# of Non-Conformances found in flight code, ground code, tools, and COTs products used (Open vs. Closed).



X

X

X

X

SWE-136

Compliance Audit Metrics

# of open non-compliances vs. # of closed non-compliances found in security scans or coding standard compliance audits.



X

X

X

X

SWE-63

Project Acceptance Metrics

# of Non-Conformances (activities not being performed)

# of Non-Conformances accepted by project

# of Non-Conformances (Open, Closed, Total)

Trends of Open vs, Closed Non-Conformances over time

X

X

X

X

X

X

SWE-087

SWE-201

SWE-039

SWE-139

Progress Tracking Metrics

Deviations of actual schedule progress vs. planned schedule progress above defined threshold

X

X

X

X

X

X

SWE-016

SWE-018

SWE-046

Progress Tracking Metrics

# of Software Requirements (e.g. Project, Application, Subsystem, System, etc.)


X

X

X

X

X

SWE-050

SWE-051

SWE-052

SWE-065b

SWE-066

SWE-194

SWE-027

Progress Tracking Metrics

# of Software Requirements that do not trace to a parent requirement   


X

X

X



SWE-033

SWE-050

SWE-051

Progress Tracking Metrics

# of architectural issues identified vs. number closed



X




SWE-057

SWE-058

SWE-143

Progress Tracking Metrics# of planned units for implementation vs. # of units tested and implemented


X

SWE-060

Progress Tracking Metrics

# of planned unit test cases vs. # of actual unit test cases completed




X



SWE-062

SWE-186

Progress Tracking Metrics

Total # of tests completed vs. # of test results evaluated and signed off





X

X

SWE-066

SWE-068

SWE-065d

SWE-159

Progress Tracking Metrics

# of Safety-Critical tests executed vs. # of Safety-Critical tests witnessed by SA




X

X

X

SWE-066

SWE-068

SWE-065d

SWE-159

SWE-062

SWE-186

Progress Tracking Metrics

# of software components (e.g. programs, modules, routines, functions, etc.) planned vs. # actually released in each build





X

X

SWE-194

SWE-077

SWE-073

SWE-036

Progress Tracking Metrics

# of Non-Conformances from reviews (Open vs. Closed; # of days Open)

X

X

X

X

X

X

SWE-134

SWE-037

SWE-039

SWE-087

SWE-088

SWE-089

SWE-143

Progress Tracking Metrics

# of Software Requirements being met via satisfactory testing vs. total # of Software Requirements




X

X

X

SWE-065d

SWE-066

SWE-192

SWE-071

Progress Tracking Metrics

# of Software Requirements without associated test cases




X

X

X

SWE-066

SWE-065b

SWE-071

Progress Tracking Metrics

# of  design issues found vs. # of design issues resolved



X

X



SWE-058

Progress Tracking Metrics

 # of planned units for implementation vs. # of units implemented and unit tested.



X

X

X


SWE-060

Risk Management Metrics

# of Risks identified in each life cycle phase (Open, Closed)

X

X

X

X

X

X

SWE-086

SWE-039

SWE-179

SWE-154

SWE-156

SWE-190

Risk Management Metrics

# of Risks by Severity (e.g., red, yellow, green) over time

X

X

X

X

X

X

SWE-032

SWE-033

SWE-039

SWE-086

SWE-154

SWE-156

SWE-179

SWE-190

SWE-191

Risk Management Metrics

# of Risks with mitigation plans vs. total # of Risks

X

X

X

X

X

X

SWE-032

SWE-033

SWE-039

SWE-086

SWE-154

SWE-156

SWE-179

SWE-190

SWE-191

Risk Management Metrics

# of Risks trending up over time

# of Risks trending down over time

X

X

X

X

X

X

SWE-032

SWE-033

SWE-039

SWE-086

SWE-154

SWE-156

SWE-179

SWE-190

SWE-191

Cybersecurity Risk Metrics

# of Cybersecurity Risks identified (Open, Closed, Severity)



X

X

X

X

SWE-154

SWE-156

Cybersecurity Risk Metrics

·of Cybersecurity Risks with Mitigations vs. # of Cybersecurity Risks identified



X

X

X

X

SWE-154

SWE-156

SWE-159

Traceability Metrics

% of traceability completed in each area: System Level requirements to Software requirements; Software Requirements to Design; Design to Code; Software Requirements to Test Procedures


X

X

X

X


SWE-052

Traceability Metrics

% of traceability completed for all hazards to software requirements and test procedures


X

X

X

X


SWE-052

Traceability Metrics

Defect trends for trace quality (# of circular traces, orphans, widows, etc.)


X





SWE-051

SWE-052

Requirements Metrics

# of detailed software requirements vs. # of estimated SLOC to be developed by the project


X

X


X


SWE-050

SWE-051

SWE-151

SWE-174

Requirements Metrics

# of incorrect, missing and incomplete requirements (i.e., # of requirements issues) vs. # of requirements issues resolved


X

X




SWE-051

SWE-053

SWE-054

Requirements Metrics

Software Requirements Volatility (# of requirements added, deleted, modified, # of TBDs over time)

 

X

X

X

X

 

SWE-200

SWE-053

Requirements Metrics

# of TBD/TBC/TBR requirements trended over time

 

 

X

X

X

X

SWE-066

Safety Metrics

# of safety-related requirement issues (Open, Closed) over time


X

X

X

X

X

SWE-039

SWE-139

SWE-205

SWE-023

SWE-134

SWE-184

SWE-052

SWE-051

SWE-058

SWE-065b

SWE-066

SWE-071

SWE-192

SWE-080

SWE-087

Safety Metrics

# of safety-related non-conformances identified by life cycle phase (over time, Open vs. Closed, # of days)


X

X

X

X

X

SWE-013

SWE-039

SWE-139

SWE-143

SWE-121

SWE-184

SWE-023

SWE-134

SWE-052

SWE-051

SWE-057

SWE-058

SWE-135

SWE-062

SWE-065a

SWE-065b

SWE-065c

SWE-065d

SWE-066

SWE-068

SWE-071

SWE-191

SWE-080

SWE-081

SWE-087

SWE-205

Reuse Metrics

# of products submitted for reuse;

# of developed products submitted for reuse vs. total # of developed products






X

SWE-147

SWE-148

Reuse Metrics

# of developed products entered in NASA Internal Sharing & Reuse System vs. total # of developed products






X

SWE-147

SWE-148

Reuse Metrics

# of products submitted for reuse vs. # of products entered into NASA Internal Sharing & Reuse Systems






X

SWE-147

SWE-148

Cybersecurity Metrics

# of Cybersecurity vulnerabilities and weaknesses identified

# of Cybersecurity vulnerabilities and weaknesses (Open, Closed, Severity)

Trending of Open vs. Closed over time

X

X

X

X

X

X

SWE-159

SWE-135

SWE-063

Cybersecurity Metrics

# and type of vulnerabilities and weaknesses identified by the project

X

X

X

X

X


SWE-135

SWE-063

Cybersecurity Metrics

# of Cybersecurity vulnerabilities and weaknesses identified by life cycle phase


X

X

X

X

X

SWE-159

SWE-135

SWE-063

Cybersecurity Metrics

# of Cybersecurity vulnerabilities and weaknesses identified vs. # resolved during Implementation




X

X

X

SWE-159

SWE-135

SWE-063

Cybersecurity Metrics

# of requirements specified relating to detection of adversarial actions vs. # of requirements related to detection of adversarial actions actually implemented




X

X

X

SWE-210

Cybersecurity Metrics

# of Non-Conformances identified in Cybersecurity coding standard compliance (Open, Closed)




X

X

X

SWE-063

SWE-135

SWE-207

Test Coverage Metrics

Software code/test coverage percentages for all identified safety-critical components (e.g., # of paths tested vs. total # of possible paths) 

Note: Metrics in bold type are required by all projects





X


SWE-066
SWE-190
SWE-189 
SWE-219

Test Coverage Metrics

Test coverage data for all identified safety-critical software components.





X


SWE-219

Test Coverage Metrics

% of code that has been executed during testing





X


SWE-060

Static Code Metrics

Document the Static Code Analysis tools used with associated Non-Conformances

·of total errors and warnings identified by the tool

# of errors and warnings evaluated vs. # of total errors and warnings identified by the tool




X



SWE-135

SWE-185

Static Code Metrics

# of Non-Conformances raised by SA vs. total # of raised Non-Conformances




X



SWE-135

SWE-185

Static Code Metrics

# of static code errors and warnings identified as “positives” vs. # of total errors and warnings identified by the tool

Total # of static code analysis "positives" vs.  # of "positives" resolved. Trend over time.

# of static code errors and warnings resolved by Severity vs. # of static code errors and warnings identified by Severity by the tool




X



SWE-135

SWE-185

Static Code Metrics

# of static code “positives” over time (Open, Closed, Severity)




X

X


SWE-135

SWE-185

Static Code Metrics

# of Cybersecurity vulnerabilities and weaknesses identified by the tool




X

X


SWE-135

SWE-185

Static Code Metrics

# of coding standard violations identified (Open, Closed, type of violation, Severity)




X



SWE-061

SWE-185

Static Code Metrics

Software cyclomatic complexity # for all identified safety-critical components




X

X


SWE-087

SWE-134

Static Code Metrics

Trend of # of total errors and warnings identified per SCA Tool, Language, and SLOC size




X



SWE-135


Static Code Metrics

Total # of static code analysis "positives" vs.  # of "positives" resolved. Trend over time.




X



SWE-135

Test Procedures Metrics

#  of Non-Conformances and risks open vs. # of Non-Conformances, risks, identified with test procedures




X

X


SWE-087

SWE-065b

SWE-071

SWE-191

Test Procedures Metrics

# of hazards with completed test procedures/cases vs. total number of hazards over time




X

X


SWE-065b

SWE-068

SWE-191

Test Procedures Metrics

# of software requirements with completed test procedures/cases over time




X

X


SWE-065b

SWE-071

SWE-066

Test Procedures Metrics

# of Non-Conformances identified when the approved, updated requirements are not reflected in test procedures




X

X


SWE-065b

SWE-071

SWE-052

SWE-066

SWE-191

Test Procedures Metrics

# of Non-Conformances identified while confirming hazard controls are verified through test plans/procedures/cases




X

X


SWE-065b

SWE-071

SWE-066

SWE-068

SWE-191

Test Procedures Metrics

# of issues and risks/corrective actions open vs. total # of issues and risks/corrective actions identified with test procedures



X

X

X


SWE-065b

Test Metrics

# of tests executed vs. # of tests completed




X

X

X

SWE-062

SWE-066

SWE-065d

SWE-068

SWE-159

SWE-190

SWE-192

SWE-055

SWE-080

SWE-191

SWE-194

SWE-211

Test Metrics

# of Non-Conformances identified during each testing phase (Open, Closed, Severity)




X

X

X

SWE-062

SWE-065d

SWE-066

SWE-068

SWE-080

SWE-159

SWE-190

SWE-191

SWE-192

SWE-194

SWE-211

Test Metrics

·of Requirements tested vs. total # of Requirements




X

X

X

SWE-062

SWE-065b

SWE-065d

SWE-066

SWE-080

SWE-191

SWE-192

SWE-194

Test Metrics

# of Hazards containing software that have been tested vs. total # of Hazards containing software




X

X

X

SWE-062

SWE-066

SWE-068

SWE-080

SWE-134

SWE-192

SWE-205

Test Metrics

# of Non-Conformances identified in models, simulations, and tools over time (Open, Closed, Severity)




X

X

X

SWE-070

Test Metrics

# of Regression test set Non-Conformances/Risks over time (Open, Closed, Severity)




X

X

X

SWE-191

Test Metrics# of risks and Non-conformances open vs. total # of risks and Non-Conformances identified with test code


XXXSWE-065c

Test Metrics

# of Requirements tested in customer environment vs. # of Requirements





X

X

SWE-055

Cybersecurity Metrics During Testing Metrics

# of Cybersecurity mitigation implementations identified from the security vulnerabilities and security weaknesses




X

X

X

SWE-154

SWE-159

SWE-191

Cybersecurity Metrics During Testing Metrics

# of Cybersecurity mitigation implementations identified with associated test procedures vs. # of Cybersecurity mitigation implementations identified




X

X

X

SWE-154

SWE-159

SWE-191

Cybersecurity Metrics During Testing Metrics

# of Cybersecurity mitigation tests completed vs. total # of Cybersecurity mitigation tests




X

X

X

SWE-159

Cybersecurity Metrics During Testing Metrics

# of Non-Conformances identified during Cybersecurity mitigation testing (Open, Closed, Severity)




X

X

X

SWE-159

Cybersecurity Metrics During Testing Metrics

Trends of Cybersecurity Non-Conformances over time




X

X

X

SWE-159

SWE-063

SWE-135

SWE-201

Test Coverage Metrics

Software code/test coverage percentages for all identified safety-critical components (e.g., # of paths tested vs. total # of possible paths)

 

 

 

X

X

X

SWE-066

SWE-190

SWE-134

SWE-189

Test Coverage Metrics

# of tests completed vs. total # of tests




X

X

X

SWE-055

SWE-062

SWE-066

SWE-065d

SWE-068

SWE-080

SWE-159

SWE-190

SWE-191

SWE-192

SWE-194

SWE-211

Test Coverage Metrics

# of detailed software requirements tested to date vs. total # of detailed software requirements




X

X

X

SWE-055

SWE-062

SWE-065b

SWE-065d

SWE-066

SWE-071

SWE-080

SWE-191

SWE-192

SWE-194

SWE-159

Test Coverage Metrics

# of safety-critical requirement verifications vs. total # of safety-critical requirement verifications completed

# of Open issues vs. # of Closed over time




X

X

X

SWE-062

SWE-066

SWE-191

SWE-192

SWE-068

Test Coverage Metrics

# of Source Lines of Code (SLOC) tested vs. total # of SLOC




X

X

X

SWE-066

SWE-134

SWE-189

SWE-190

SWE-219

Build/Release Content Metrics

# of software units planned vs. # actually built



X

X

X

X

SWE-063

SWE-194

Build/Release Content Metrics

# of planned software requirements implemented in each build vs. # of actual software requirements implemented in each build




X

X

X

SWE-063

SWE-194

Build/Release Content Metrics

# of Non-Conformances identified in release documentation (Open, Closed)




X

X

X

SWE-063

SWE-077

SWE-084

SWE-085

Operations/ Maintenance Metrics

# of Non-Conformances identified in the software after delivery






X

SWE-195

Other Metrics

Measures relating to status and performance as identified in other requirements. (Schedule deviations, closure of corrective actions, product and process audit results, peer review result, etc)



X

X

X

X

SWE-093

Other Metrics

# of requirements being met, any TBD/TBC/TBR requirements, or any requirements that are not being met

Performance and functionality measures (Schedule deviations, closure of corrective actions, product and process audit results, peer review results, requirements volatility, # of requirements satisfactorily tested vs. total # of requirements, etc.)



X

X

X

X

SWE-199

Other Metrics

# of findings open vs. # of findings identified

X

X

X

X

X


SWE-082

Other Metrics

Trends of Open vs. Closed Non-Conformances change reports over time.

X

X

X

X

X


SWE-065d

IV&V Metrics (Kept by IV&V)

# of IV&V Non-Conformances/issues,/risks (Open, Closed, accepted by the project, severity, category of Non-Conformance (e.g., requirements severity, category (requirements, design, code, test, documentation, etc.))

Kept by IV&V team, not SA

X

X

X

X

X

X

SWE-179


Coverage Metrics

Code coverage data: % of code that has been executed during testing




X

X

X

SWE-060

Coverage Metrics

Test coverage data for all identified safety-critical software components.




 X

X

X

SWE-219

Old

...

Metrics Type

...

Measurements

...

Plan

...

Reqt

...

Des

...

Imp

...

Test

...

Del

...

Associated SWE Reqt #

...

Peer Review Metrics

...

# of peer reviews performed vs. # of peer reviews planned

...

X

...

X

...

X

...

X

...

X

...

SWE-016

SWE-087

SWE-089

...

Peer Review Metrics

...

# of Non-Conformances identified in each peer review

...

X

...

X

...

X

...

X

...

X

...

SWE-087

SWE-089

...

Peer Review Metrics

...

# of Non-Conformances identified by software assurance during each peer review

...

X

...

X

...

X

...

X

...

X

...

SWE-087

SWE-088

SWE-089

...

Peer Review Metrics

...

Total # of peer review Non-Conformances (Open, Closed)

...

X

...

X

...

X

...

X

...

X

...

SWE-087

SWE-088

SWE-089

...

Peer Review Metrics

...

Preparation time each review participant spent preparing for the review

...

X

...

X

...

X

...

X

...

X

...

SWE-088

SWE-089

...

Peer Review Metrics

...

Time required to close review Non-Conformances

...

X

...

X

...

X

...

X

...

X

...

SWE-087

SWE-088

SWE-089

...

Peer Review Metrics

...

# of peer review participants vs. total # invited

...

X

...

X

...

X

...

X

...

X

...

SWE-088

SWE-089

...

Peer Review Metrics

...

# of peer review Non-Conformances per work product vs. # of peer reviewers

...

X

...

X

...

X

...

X

...

X

...

SWE-088

SWE-089

...

Peer Review Audit Metrics

...

# of audit Non-Conformances per peer review audit

...

X

...

X

...

X

...

X

...

X

...

SWE-088

...

Peer Review Audit Metrics

...

# of Peer Review Audits planned vs. # of Peer Review Audits performed

...

X

...

X

...

X

...

X

...

X

...

SWE-016

SWE-087

SWE-088

...

Peer Review Audit Metrics

...

Trends on non-conformances from audits (Open, Closed, Life cycle Phase)

...

X

...

X

...

X

...

X

...

X

...

SWE-088

SWE-089

...

Peer Review Audit Metrics

...

Time required to close peer review audit Non-Conformances

...

X

...

X

...

X

...

X

...

X

...

SWE-088

SWE-089

...

Peer Review Audit Metrics

...

Preparation time each audit participant spent preparing for audit

...

X

...

X

...

X

...

X

...

X

...

SWE-088

SWE-089

...

Problem/Change Report Status Metrics

...

Total # of Non-Conformances over time (Open, Closed, # of days Open, and Severity of Open)

# of Non-Conformances in current reporting period (Open, Closed, Severity)

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-062

SWE-065c

SWE-065d

SWE-068

SWE-202

SWE-203

SWE-204

...

Problem/Change Report Status Metrics

...

# of safety-related Non-Conformances

...

X

...

X

...

X

...

X

...

X

...

SWE-203

SWE-068

SWE-071

...

Problem/Change Report Status Metrics

...

Trend of Open vs. Closed Non-Conformances over time

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-053

SWE-054

SWE-065

...

Problem/Change Report Status Metrics

...

Trend of change status over time (# of changes approved, # in implementation, # in test, # closed)

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-018

SWE-053

SWE-080

...

Problem/Change Report Status Metrics

...

# of Non-Conformances identified in embedded COTS, GOT, MOTS, OSS, or reused components in ground or flight software vs. # of Non-Conformances closed

...

X

...

X

...

X

...

SWE-136

SWE-202

SWE-203

SWE-211

...

Problem/Change Report Status Metrics

...

# of Non-Conformances identified in source code products used (Open, Closed)

...

X

...

X

...

X

...

SWE-202

SWE-203

...

Problem/Change Report Status Metrics

...

# of software Non-Conformances at each Severity level for each software configuration item (Open, Closed)

        Bold type indicates a required item

...

 

...

 

...

 

...

X

...

X

...

X

...

SWE-202

...

Problem/Change Report Status Metrics

...

# of Closed action items vs. # of Open action items

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-062 SWE-62c

SWE-065d

...

Problem/Change Report Status Metrics

...

# of Root Cause Analyses performed;

# of Non-Conformances identified by each root cause analysis

...

X

...

X

...

X

...

X

...

SWE-204

...

SA Corrective Action (CA) metrics (Issues, Risks)

...

# of Corrective Actions (CAs) raised by SA vs. total #

       Attributes (Type, Severity, # of days Open, Life cycle Phase Found)

      State (Open, In work, Closed)

      Trends of CA Open vs. Closures over time

...

 X

...

X

...

X

...

X

...

X

...

X

...

SWE-024

SWE-204

SWE-054

...

SA Corrective Action (CA) metrics (Issues, Risks)

...

Trend the # of inconsistencies or corrective actions identified, and # closed.

...

X

...

X

...

X

...

X

...

X

...

SWE-054

SWE-024

SWE-204

...

Process Improvement Metrics

...

# of software work product Non-Conformances identified by life cycle phase over time

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-013

SWE-022

SWE-024

SWE-039

SWE-051

SWE-054

SWE-057

SWE-058

SWE-062

SWE-065b

SWE-065c

SWE-065d

SWE-068

SWE-071

SWE-075

SWE-079

SWE-084

SWE-086

SWE-087

SWE-125

SWE-134

SWE-139

SWE-146

SWE-157

SWE-159

SWE-184

SWE-185

SWE-187

SWE-191

SWE-194

SWE-201

SWE-204

SWE-205

...

Process Improvement Metrics

...

# of software process Non-Conformances by life cycle phase over time

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-032

SWE-039

SWE-061

SWE-077

SWE-080

SWE-082

SWE-085

SWE-086

SWE-088

SWE-139

SWE-195

SWE-204

...

Process Improvement Metrics

...

Identify the specific requirements in NASA-STD-8739.8 that are being tailored by the projects (*organizational metric)

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-121

SWE-125

SWE-013

SWE-176

...

Process Improvement Metrics

...

# of projects tailoring each requirement (*organizational measure)

% of requirements tailored per project (*organizational measure)

...

X

...

X

...

X

...

X

...

SWE-125

SWE-013

SWE-121

...

Process Improvement Metrics

...

% of Total Source Code for each Software Classification (*organizational measure)

...

X

...

X

...

X

...

SWE-020

SWE-176

SWE-087

...

Cost/ Effort Metrics

...

Planned SA resource allocation vs. actual SA resource allocation 

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-015

SWE-174

SWE-151

...

Cost/ Effort Metrics

...

Comparison of initial SA cost estimates vs. final cost (capturing assumptions and differences)

...

X

...

X

...

SWE-174

SWE-015

SWE-151

...

Cost/ Effort Metrics

...

Trend SA cost estimates throughout life cycle

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-174

...

Training Metrics

...

% of required training completed for each of the project SA personnel

...

X

...

X

...

X

...

X

...

SWE-017

...

Training Metrics

...

% of project personnel that have completed project specific training against the planned training schedule

...

X

...

X

...

X

...

X

...

X

...

SWE-017

...

Compliance Audit Metrics

...

# of Compliance Audits planned vs. # of Compliance Audits performed

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-024

SWE-039

SWE-139

SWE-016

SWE-032

SWE-077

SWE-195

SWE-082

SWE-084

SWE-085

SWE-086

SWE-088

SWE-079 SWE-201

...

Compliance Audit Metrics

# of Open vs. Closed Audit Non-Conformances over time

# of Non-Conformances per audit (including findings from process and compliance audits, process maturity)

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-022

SWE-024

SWE-039

SWE-139

SWE-016

SWE-032

SWE-077

SWE-079

SWE-195

SWE-082

SWE-084

SWE-085

SWE-086

SWE-088

SWE-201

...

Compliance Audit Metrics

...

# of Non-Conformances identified in plans (e.g., SMPs, SDPs, CM Plans, SA Plans, Safety Plans, Test Plans)

...

X

...

X

...

X

...

X

...

X

...

SWE-024

SWE-013

SWE-075

SWE-079

SWE-139

SWE-071

...

Compliance Audit Metrics

...

#  of Non-Conformances identified in the software Configuration Management Plan

Trends of # Open vs. # Closed over time

...

X

...

X

...

X

...

X

...

X

...

SWE-079

...

Compliance Audit Metrics

...

#  of Configuration Management Audits conducted by the project – Planned vs. Actual

...

X

...

X

...

X

...

X

...

X

...

SWE-082

SWE-077

SWE-084

...

Compliance Audit Metrics

...

# of Non-Conformances per audit (including findings from process and compliance audits, process maturity)

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-024

SWE-039

SWE-139

SWE-016

SWE-032

SWE-077

SWE-195

SWE-082

SWE-084

SWE-085

SWE-086

SWE-088

SWE-079

...

Compliance Audit Metrics

...

# of process Non-Conformances (e.g., activities not performed) identified by SA vs. # accepted by the project

Trends of # Open vs. # Closed over time

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-016

SWE-039

SWE-032

SWE-077

SWE-195

SWE-082

SWE-084

SWE-085

SWE-086

SWE-088

...

Compliance Audit Metrics

...

# of Non-Conformances found in flight code, ground code, tools, and COTs products used (Open vs. Closed).

...

X

...

X

...

X

...

X

...

SWE-136

...

Compliance Audit Metrics

...

# of open non-compliances vs. # of closed non-compliances found in security scans or coding standard compliance audits.

...

X

...

X

...

X

...

X

...

SWE-63

...

Project Acceptance Metrics

...

# of Non-Conformances (activities not being performed)

# of Non-Conformances accepted by project

# of Non-Conformances (Open, Closed, Total)

Trends of Open vs, Closed Non-Conformances over time

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-087

SWE-201

SWE-039

SWE-139

...

Progress Tracking Metrics

...

Deviations of actual schedule progress vs. planned schedule progress above defined threshold

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-016

SWE-018

SWE-046

...

Progress Tracking Metrics

...

# of Software Requirements (e.g. Project, Application, Subsystem, System, etc.)

...

X

...

X

...

X

...

X

...

X

...

SWE-050

SWE-051

SWE-052

SWE-065b

SWE-066

SWE-194

SWE-027

...

Progress Tracking Metrics

...

# of Software Requirements that do not trace to a parent requirement   

...

X

...

X

...

X

...

SWE-033

SWE-050

SWE-051

...

Progress Tracking Metrics

...

# of architectural issues identified vs. number closed

...

X

...

SWE-057

SWE-058

SWE-143

...

Progress Tracking Metrics

...

# of planned unit test cases vs. # of actual unit test cases completed

...

X

...

SWE-062

SWE-186

...

Progress Tracking Metrics

...

Total # of tests completed vs. # of test results evaluated and signed off

...

X

...

X

...

SWE-066

SWE-068

SWE-065d

SWE-159

...

Progress Tracking Metrics

...

# of Safety-Critical tests executed vs. # of Safety-Critical tests witnessed by SA

...

X

...

X

...

X

...

SWE-066

SWE-068

SWE-065d

SWE-159

SWE-062

SWE-186

...

Progress Tracking Metrics

...

# of software components (e.g. programs, modules, routines, functions, etc.) planned vs. # actually released in each build

...

X

...

X

...

SWE-194

SWE-077

SWE-073

SWE-036

...

Progress Tracking Metrics

...

# of Non-Conformances from reviews (Open vs. Closed; # of days Open)

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-134

SWE-037

SWE-039

SWE-087

SWE-088

SWE-089

SWE-143

...

Progress Tracking Metrics

...

# of Software Requirements being met via satisfactory testing vs. total # of Software Requirements

...

X

...

X

...

X

...

SWE-065d

SWE-066

SWE-192

SWE-071

...

Progress Tracking Metrics

...

# of Software Requirements without associated test cases

...

X

...

X

...

X

...

SWE-066

SWE-065b

SWE-071

...

Progress Tracking Metrics

...

# of  design issues found vs. # of design issues resolved

...

X

...

X

...

SWE-058

...

Progress Tracking Metrics

...

 # of planned units for implementation vs. # of units implemented and unit tested.

...

X

...

X

...

X

...

SWE-060

...

Risk Management Metrics

...

# of Risks identified in each life cycle phase (Open, Closed)

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-086

SWE-039

SWE-179

SWE-154

SWE-156

SWE-190

...

Risk Management Metrics

...

# of Risks by Severity (e.g., red, yellow, green) over time

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-032

SWE-033

SWE-039

SWE-086

SWE-154

SWE-156

SWE-179

SWE-190

SWE-191

...

Risk Management Metrics

...

# of Risks with mitigation plans vs. total # of Risks

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-032

SWE-033

SWE-039

SWE-086

SWE-154

SWE-156

SWE-179

SWE-190

SWE-191

...

Risk Management Metrics

...

# of Risks trending up over time

# of Risks trending down over time

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-032

SWE-033

SWE-039

SWE-086

SWE-154

SWE-156

SWE-179

SWE-190

SWE-191

...

Cybersecurity Risk Metrics

...

X

...

X

...

X

...

X

...

SWE-154

SWE-156

...

Cybersecurity Risk Metrics

...

·of Cybersecurity Risks with Mitigations vs. # of Cybersecurity Risks identified

...

X

...

X

...

X

...

X

...

SWE-154

SWE-156

SWE-159

...

Traceability Metrics

...

% of traceability completed in each area: System Level requirements to Software requirements; Software Requirements to Design; Design to Code; Software Requirements to Test Procedures

...

X

...

X

...

X

...

X

...

SWE-052

...

Traceability Metrics

...

% of traceability completed for all hazards to software requirements and test procedures

...

X

...

X

...

X

...

X

...

SWE-052

...

Traceability Metrics

...

Defect trends for trace quality (# of circular traces, orphans, widows, etc.)

...

X

...

SWE-051

SWE-052

...

Requirements Metrics

...

# of detailed software requirements vs. # of estimated SLOC to be developed by the project

...

X

...

X

...

X

...

SWE-050

SWE-051

SWE-151

SWE-174

...

Requirements Metrics

...

# of incorrect, missing and incomplete requirements (i.e., # of requirements issues) vs. # of requirements issues resolved

...

X

...

X

...

SWE-051

SWE-053

SWE-054

...

Requirements Metrics

...

Software Requirements Volatility (# of requirements added, deleted, modified, # of TBDs over time)

...

 

...

X

...

X

...

X

...

X

...

 

...

SWE-200

SWE-053

...

Requirements Metrics

...

# of TBD/TBC/TBR requirements trended over time

...

 

...

 

...

X

...

X

...

X

...

X

...

SWE-066

...

Safety Metrics

...

# of safety-related requirement issues (Open, Closed) over time

...

X

...

X

...

X

...

X

...

X

...

SWE-039

SWE-139

SWE-205

SWE-023

SWE-134

SWE-184

SWE-052

SWE-051

SWE-058

SWE-065b

SWE-066

SWE-071

SWE-192

SWE-080

SWE-087

...

Safety Metrics

...

# of safety-related non-conformances identified by life cycle phase (over time, Open vs. Closed, # of days)

...

X

...

X

...

X

...

X

...

X

...

SWE-013

SWE-039

SWE-139

SWE-143

SWE-121

SWE-184

SWE-023

SWE-134

SWE-052

SWE-051

SWE-057

SWE-058

SWE-135

SWE-062

SWE-065a

SWE-065b

SWE-065c

SWE-065d

SWE-066

SWE-068

SWE-071

SWE-191

SWE-080

SWE-081

SWE-087

SWE-205

...

Reuse Metrics

...

# of products submitted for reuse;

# of developed products submitted for reuse vs. total # of developed products

...

X

...

SWE-147

SWE-148

...

Reuse Metrics

...

# of developed products entered in NASA Internal Sharing & Reuse System vs. total # of developed products

...

X

...

SWE-147

SWE-148

...

Reuse Metrics

...

# of products submitted for reuse vs. # of products entered into NASA Internal Sharing & Reuse Systems

...

X

...

SWE-147

SWE-148

...

Cybersecurity Metrics

...

# of Cybersecurity vulnerabilities and weaknesses identified

# of Cybersecurity vulnerabilities and weaknesses (Open, Closed, Severity)

Trending of Open vs. Closed over time

...

X

...

X

...

X

...

X

...

X

...

X

...

SWE-159

SWE-135

SWE-063

...

Cybersecurity Metrics

...

# and type of vulnerabilities and weaknesses identified by the project

...

X

...

X

...

X

...

X

...

X

...

SWE-135

SWE-063

...

Cybersecurity Metrics

...

# of Cybersecurity vulnerabilities and weaknesses identified by life cycle phase

...

X

...

X

...

X

...

X

...

X

...

SWE-159

SWE-135

SWE-063

...

Cybersecurity Metrics

...

# of Cybersecurity vulnerabilities and weaknesses identified vs. # resolved during Implementation

...

X

...

X

...

X

...

SWE-159

SWE-135

SWE-063

...

Cybersecurity Metrics

...

# of requirements specified relating to detection of adversarial actions vs. # of requirements related to detection of adversarial actions actually implemented

...

X

...

X

...

X

...

SWE-210

...

Cybersecurity Metrics

...

# of Non-Conformances identified in Cybersecurity coding standard compliance (Open, Closed)

...

X

...

X

...

X

...

SWE-063

SWE-135

SWE-207

...

Test Coverage Metrics

...

Software code/test coverage percentages for all identified safety-critical components (e.g., # of paths tested vs. total # of possible paths) 

Note: Metrics in bold type are required by all projects

...

X

...

SWE-066
SWE-190
SWE-189 
SWE-219

...

Test Coverage Metrics

...

Test coverage data for all identified safety-critical software components.

...

X

...

SWE-219

...

Test Coverage Metrics

...

% of code that has been executed during testing

...

X

...

SWE-060

...

Static Code Metrics

...

Document the Static Code Analysis tools used with associated Non-Conformances

·of total errors and warnings identified by the tool

# of errors and warnings evaluated vs. # of total errors and warnings identified by the tool

...

X

...

SWE-135

SWE-185

...

Static Code Metrics

...

# of Non-Conformances raised by SA vs. total # of raised Non-Conformances

...

X

...

SWE-135

SWE-185

...

Static Code Metrics

...

# of static code errors and warnings identified as “positives” vs. # of total errors and warnings identified by the tool

Total # of static code analysis "positives" vs.  # of "positives" resolved. Trend over time.

# of static code errors and warnings resolved by Severity vs. # of static code errors and warnings identified by Severity by the tool

...

X

...

SWE-135

SWE-185

...

Static Code Metrics

...

# of static code “positives” over time (Open, Closed, Severity)

...

X

...

X

...

SWE-135

SWE-185

...

Static Code Metrics

...

# of Cybersecurity vulnerabilities and weaknesses identified by the tool

...

X

...

X

...

SWE-135

SWE-185

...

Static Code Metrics

...

# of coding standard violations identified (Open, Closed, type of violation, Severity)

...

X

...

SWE-061

SWE-185

...

Static Code Metrics

...

Software cyclomatic complexity # for all identified safety-critical components

...

X

...

X

...

SWE-087

SWE-134

...

Static Code Metrics

...

Trend of # of total errors and warnings identified per SCA Tool, Language, and SLOC size

...

X

SWE-135

...

Static Code Metrics

...

Total # of static code analysis "positives" vs.  # of "positives" resolved. Trend over time.

...

X

...

SWE-135

...

Test Procedures Metrics

...

#  of Non-Conformances and risks open vs. # of Non-Conformances, risks, identified with test procedures

...

X

...

X

...

SWE-087

SWE-065b

SWE-071

SWE-191

...

Test Procedures Metrics

...

# of hazards with completed test procedures/cases vs. total number of hazards over time

...

X

...

X

...

SWE-065b

SWE-068

SWE-191

...

Test Procedures Metrics

...

# of software requirements with completed test procedures/cases over time

...

X

...

X

...

SWE-065b

SWE-071

SWE-066

...

Test Procedures Metrics

...

# of Non-Conformances identified when the approved, updated requirements are not reflected in test procedures

...

X

...

X

...

SWE-065b

SWE-071

SWE-052

SWE-066

SWE-191

...

Test Procedures Metrics

...

# of Non-Conformances identified while confirming hazard controls are verified through test plans/procedures/cases

...

X

...

X

...

SWE-065b

SWE-071

SWE-066

SWE-068

SWE-191

...

Test Procedures Metrics

...

# of issues and risks/corrective actions open vs. total # of issues and risks/corrective actions identified with test procedures

...

X

...

X

...

X

...

SWE-065b

...

Test Metrics

...

# of tests executed vs. # of tests completed

...

X

...

X

...

X

...

SWE-062

SWE-066

SWE-065d

SWE-068

SWE-159

SWE-190

SWE-192

SWE-055

SWE-080

SWE-191

SWE-194

SWE-211

...

Test Metrics

...

# of Non-Conformances identified during each testing phase (Open, Closed, Severity)

...

X

...

X

...

X

...

SWE-062

SWE-065d

SWE-066

SWE-068

SWE-080

SWE-159

SWE-190

SWE-191

SWE-192

SWE-194

SWE-211

...

Test Metrics

...

·of Requirements tested vs. total # of Requirements

...

X

...

X

...

X

...

SWE-062

SWE-065b

SWE-065d

SWE-066

SWE-080

SWE-191

SWE-192

SWE-194

...

Test Metrics

...

# of Hazards containing software that have been tested vs. total # of Hazards containing software

...

X

...

X

...

X

...

SWE-062

SWE-066

SWE-068

SWE-080

SWE-134

SWE-192

SWE-205

...

Test Metrics

...

# of Non-Conformances identified in models, simulations, and tools over time (Open, Closed, Severity)

...

X

...

X

...

X

...

SWE-070

...

Test Metrics

...

# of Regression test set Non-Conformances/Risks over time (Open, Closed, Severity)

...

X

...

X

...

X

...

SWE-191

...

Test Metrics

...

# of Requirements tested in customer environment vs. # of Requirements

...

X

...

X

...

SWE-055

...

Cybersecurity Metrics During Testing Metrics

...

# of Cybersecurity mitigation implementations identified from the security vulnerabilities and security weaknesses

...

X

...

X

...

X

...

SWE-154

SWE-159

SWE-191

...

Cybersecurity Metrics During Testing Metrics

...

# of Cybersecurity mitigation implementations identified with associated test procedures vs. # of Cybersecurity mitigation implementations identified

...

X

...

X

...

X

...

SWE-154

SWE-159

SWE-191

...

Cybersecurity Metrics During Testing Metrics

...

# of Cybersecurity mitigation tests completed vs. total # of Cybersecurity mitigation tests

...

X

...

X

...

X

...

SWE-159

...

Cybersecurity Metrics During Testing Metrics

...

# of Non-Conformances identified during Cybersecurity mitigation testing (Open, Closed, Severity)

...

X

...

X

...

X

...

SWE-159

...

Cybersecurity Metrics During Testing Metrics

...

Trends of Cybersecurity Non-Conformances over time

...

X

...

X

...

X

...

SWE-159

SWE-063

SWE-135

SWE-201

...

Test Coverage Metrics

...

Software code/test coverage percentages for all identified safety-critical components (e.g., # of paths tested vs. total # of possible paths)

...

 

...

 

...

 

...

X

...

X

...

X

...

SWE-066

SWE-190

SWE-134

SWE-189

...

Test Coverage Metrics

...

# of tests completed vs. total # of tests

...

X

...

X

...

X

...

SWE-055

SWE-062

SWE-066

SWE-065d

SWE-068

SWE-080

SWE-159

SWE-190

SWE-191

SWE-192

SWE-194

SWE-211

...

Test Coverage Metrics

...

# of detailed software requirements tested to date vs. total # of detailed software requirements

...

X

...

X

...

X

...

SWE-055

SWE-062

SWE-065b

SWE-065d

SWE-066

SWE-071

SWE-080

SWE-191

SWE-192

SWE-194

SWE-159

...

Test Coverage Metrics

...

# of safety-critical requirement verifications vs. total # of safety-critical requirement verifications completed

# of Open issues vs. # of Closed over time

...

X

...

X

...

X

...

SWE-062

SWE-066

SWE-191

SWE-192

SWE-068

...

Test Coverage Metrics

...

# of Source Lines of Code (SLOC) tested vs. total # of SLOC

...

X

...

X

...

X

...

SWE-066

SWE-134

SWE-189

SWE-190

SWE-219

...

Build/Release Content Metrics

...

# of software units planned vs. # actually built

...

X

...

X

...

X

...

X

...

SWE-063

SWE-194

...

Build/Release Content Metrics

...

# of planned software requirements implemented in each build vs. # of actual software requirements implemented in each build

...

X

...

X

...

X

...

SWE-063

SWE-194

...

Build/Release Content Metrics

...

# of Non-Conformances identified in release documentation (Open, Closed)

...

X

...

X

...

X

...

SWE-063

SWE-077

SWE-084

SWE-085

...

Operations/ Maintenance Metrics

...

# of Non-Conformances identified in the software after delivery

...

X

...

SWE-195

...

Other Metrics

...

Measures relating to status and performance as identified in other requirements. (Schedule deviations, closure of corrective actions, product and process audit results, peer review result, etc)

...

X

...

X

...

X

...

X

...

SWE-093

...

Other Metrics

...

# of requirements being met, any TBD/TBC/TBR requirements, or any requirements that are not being met

Performance and functionality measures (Schedule deviations, closure of corrective actions, product and process audit results, peer review results, requirements volatility, # of requirements satisfactorily tested vs. total # of requirements, etc.)

...

X

...

X

...

X

...

X

...

SWE-199

...

Other Metrics

...

# of findings open vs. # of findings identified

...

X

...

X

...

X

...

X

...

X

...

SWE-082

...

Other Metrics

...

Trends of Open vs. Closed Non-Conformances change reports over time.

...

X

...

X

...

X

...

X

...

X

...

SWE-065d

...

IV&V Metrics (Kept by IV&V)

...

# of IV&V Non-Conformances/issues,/risks (Open, Closed, accepted by the project, severity, category of Non-Conformance (e.g., requirements severity, category (requirements, design, code, test, documentation, etc.))

Kept by IV&V team, not SA

...

X

...

X

...

X

...

X

...

X

...

X

SWE-179

...

Coverage Metrics

...

Code coverage data: % of code that has been executed during testing

...

X

...

X

...

X

...

SWE-060

...

Coverage Metrics

...

Test coverage data for all identified safety-critical software components.

...

 X

...

X

...

X

...

SWE-219




Page Editor: Tim Crumbley
NASA Official: Tim Crumbley
Accessibility
NASA Software Engineering Handbook - A service of the NASA Office of the Chief Engineer
NASA Privacy Statement
______________________________________________________________________________