Comment:
Migration of unmigrated content due to installation of a new plugin
Tabsetup
0
1. The Requirement
1
2. Rationale
2
3. Guidance
3
4. Small Projects
4
5. Resources
5
6. Lessons Learned
6
7. Software Assurance
Div
id
tabs-1
1. Requirements
Excerpt
5.5.1 The project manager shall track and maintain software non-conformances (including defects in tools and appropriate ground software).
1.1 Notes
NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.
1.2 History
Expand
title
Click here to view the history of this requirement: SWE-201 History
Include Page
SITE:SWE-201 History
SITE:SWE-201 History
1.3 Applicability Across Classes
Applicable c
a
1
b
1
csc
1
c
1
d
1
dsc
1
e
0
f
1
g
0
h
0
Div
id
tabs-2
2. Rationale
To make sure that all software non-conformances are addressed. Managing and tracking non-conformances, software problem reports, or software issues are critical steps to ensuring that software defects are flagged and handled properly.
Div
id
tabs-3
3. Guidance
A Software non-conformance is any difference between plans, specifications, and software functionality that remains in the products after software delivery. Even with a rigorous development process and thorough testing, some defects typically remain in the delivered products resulting in non-conformances.
Maintaining records of non-conformances is essential to enable future software reuse or software upgrades. Engineers who plan to reuse existing software products need thorough documentation of the root cause of anomalous behavior, documentation discrepancies, operational constraints, and operational workarounds. Records of non-conformances can also be used to analyze the reliability and quality of delivered software products. Non-conformance records are also useful in pinpointing specific software components or subsystems with excessive numbers of defects that remain in the delivered software.
Software Change Tracking Tools can be used to document non-conformances, but the non-conformance data must be accessible to current and potential future users of software products. The following is a suggested list of items that should be included in non-conformance tracking data:
Date non-conformance was discovered
How the non-conformance was discovered
The severity of non-conformance (Refer to SWE-202 for a discussion of Severity Levels)
Configuration information (i.e., Version information for the software, version information for any tools used, document revision number, etc…)
Contact information for responsible engineers
Location of any supporting data
Recording, tracking, and analyzing the root cause of non-conformances is a best practice. (Refer to SWE-204 for a discussion of Root Cause Analysis.)
Be cautious about closing overlapping software change tickets to ensure that the full scope of all associated software change tickets is addressed via retest. Be careful about missing any unique elements to the individual software change ticket.
Any aspects of control board decisions that are modified must be re-approved by the board (e.g., Impacted artifacts that need updating).
Div
id
tabs-4
4. Small Projects
No additional guidance is available for small projects.
Div
id
tabs-5
5. Resources
5.1 References
refstable
Show If
group
confluence-users
Panel
titleColor
red
title
Visible to editors only
Enter the necessary modifications to be made in the table below:
SWEREFs to be added
SWEREFS to be deleted
SWEREFs NOT called out in text but listed as germane: none
SWEREFs called out in text: none
5.2 Tools
Include Page
Tools Table Statement
Tools Table Statement
Div
id
tabs-6
6. Lessons Learned
6.1 NASA Lessons Learned
No Lessons Learned have currently been identified for this requirement.
6.2 Other Lessons Learned
Software Change Tickets
Any tests (formal or informal) that fail should be rerun and verified before software change tickets are closed, in the original environment, or as close to it as possible. Preferably this would be done with the original author of the software change ticket but with appropriate control board approval.
Be cautious about closing overlapping software change tickets to ensure that the full scope of all associated software change tickets is addressed via retest. Be careful about missing any unique elements to the individual software change ticket.
Div
id
tabs-7
7. Software Assurance
Excerpt Include
SWE-201 - Software Non-Conformances
SWE-201 - Software Non-Conformances
7.1 Tasking for Software Assurance
Confirm that all software non-conformances are recorded and tracked to resolution.
Confirm that accepted non-conformances include the rationale for the non-conformance.
7.2 Software Assurance Products
List of non-conformances for which SA has verified the closure.
Note
title
Objective Evidence
Software defect or problem reporting data
Software configuration management data
Software assurance audit results on the change management process
Software milestone results
Software version description documents
Software control board data or presentations
Expand
title
Definition of objective evidence
Include Page
SITE:Definition of Objective Evidence
SITE:Definition of Objective Evidence
7.3 Metrics
Trends of Cybersecurity Non-Conformances over time
# of Non-Conformances (activities not being performed)
# of Non-Conformances accepted by the project
# of Non-Conformances (Open, Closed, Total)
Trends of Non-Conformances over time
# of Open vs. Closed Audit Non-Conformances over time
Trends of # of Non-Conformances from audits over time (Include counts from process and standards audits and work product audits.)
# of software work product Non-Conformances identified by life-cycle phase over time
7.4 Guidance
One of the key items in assuring that software quality is achieved in a system is the management of non-conformances that arise as the software system goes through the different levels of testing. Software assurance needs to keep a close watch on activities during those periods by either witnessing the tests or reviewing test reports to make sure all non-conformances (including errors, discrepancies, as well as other types of non-conformances) are being handled properly.
Activities that software assurance needs to perform associated with non-conformances:
Confirm all discrepancies are recorded in the discrepancy reporting system (SWE-201, Task 1)
When discrepancies are discovered while the test is being witnessed, check to see that the discrepancies are noted on the test results. If the discrepancy is not also recorded in the discrepancy system during the test, look at the discrepancy system after the test to verify that the discrepancy has been recorded. If the test was not witnessed, examine the resulting test results and check that any discrepancies noted are recorded also in the discrepancy system.
Confirm discrepancies recorded include defects in tools and ground systems (SWE-201)
The same process as above can be used to check those discrepancies in tools and ground systems being tested are recorded. If the discrepancies in the tools or ground systems are discovered during the testing of some other software, check to see that they are recorded in the test results where they were discovered and then verify that they have been recorded in the discrepancy system.
Confirm discrepancies recorded include discrepancies received from vendors of COTS, MOTS, GOTS, OSS, reused software, as well as discrepancies from this software found during testing of the project software (as per SWE-202)
Review the vendor websites of the COTS, MOTS, GOTS, OSS, reused being used to get a list of the discrepancies that have been documented. Check to see that these are also listed in the project discrepancy system. For GOTS or reused software, it may be necessary to contact the developers of this software to obtain a list of the known discrepancies. Also check to see if any discrepancies in the COTS, MOTS, GOTS, OSS, or reused software are discovered during the testing of project software, verify that those discrepancies have been added to the project discrepancy list and that the vendor/developers have been notified.
Confirm that all recorded non-conformances have associated severity levels - correctly assigned (as per SWE-202, Task 2,3)
When reviewing the discrepancies documented in the project discrepancy system, check that the appropriate severity level for each discrepancy has been recorded and that the severity level assigned meets the project-defined severity definition.
Confirm that each non-conformance does not exist elsewhere in the system (SWE-201, Task 2)
Work with the project developer to identify areas where the code may be repeated or is similar, or where a similar error in logic or design may have caused a similar discrepancy and then examine those areas to determine if the same discrepancies exist (or work with the developer to assure that he/she has checked those areas.
If a non-conformance is accepted for implementation of a correction, confirm that the rationale for the decision is recorded (SWE-201-Task 3)
Review the non-compliance records in the discrepancy system after the non-conformance has been approved for implementation and check to see that the decision rationale has been recorded.
Track each non-conformance to closure
Periodically, review the project discrepancy system to assure that all discrepancies are being addressed promptly. Discrepancies approved for implementation must be implemented and then tested, including regression testing. Other discrepancies should eventually have some final resolution like a work-around or a decision that the discrepancy doesn't need to be fixed.
Review metrics on the non-conformances, including:
Number of non-conformances at each severity level (SWE-202)
Total number of non-conformances/discrepancies (SWE-201)
Number of non-conformances/discrepancies closed (SWE-201)
Number of non-conformances/discrepancies open during this period (SWE-201)
Age of non-conformance/discrepancy (SWE-201)
The severity of each non-conformance/discrepancy (SWE-201)
Be cautious about closing overlapping software change tickets to ensure that the full scope of all associated software change tickets is addressed via retest. Be careful about missing any unique elements to the individual software change ticket.
Any aspects of control board decisions that are modified must be re-approved by the board (e.g., Impacted artifacts that need updating).