1. Requirements

5.1.8 The project manager shall establish and implement procedures for the storage, handling, delivery, release, and maintenance of deliverable software products.  

1.1 Notes

NPR 7150.2, NASA Software Engineering Requirements, does not include any notes for this requirement.

1.2 History

1.3 Applicability Across Classes

Key:    - Applicable | - Not Applicable

2. Rationale

Given that software is considered a significant Agency investment, it is important to ensure that the delivered software is created and maintained from a controlled repository. Configuration management (CM) processes and controls provide the rigor and organization necessary for developers and their customers to have confidence that all changes to the code and documents are included in the released products. It is also important that the released product is stored, maintained, and delivered following a repeatable, controlled process.

3. Guidance

3.1 Definitions

Example: source code, code for execution, or multiple software assets packaged into an internal production release and tested for a target platform, test release 

3.2 Release Management Procedures

As with other CM activities, the CM plan includes the plans and references the procedures for software release management.

When developing procedures for release management, address all of the following:

• Preparation of the release package.
• Creation and delivery of the release package.
• Storage and maintenance of the release package.

Release management procedures may vary depending on the recipient of the release. Internal releases, such as baselines released for testing, will most likely not require the same set of release activities and considerations as formal releases to external customers.

See also SWE-077 - Deliver Software Products, SWE-194 - Delivery Requirements Verification, 

3.2.1 Preparation of the Release Package

The SMA (Safety and Mission Assurance) Technical Excellence Program (STEP) Level 2 Software Configuration Management and Data Management course taught by the Westfall Team ³⁴³ provides a good list of release planning and scheduling activities.

A checklist of activities to complete may be useful as part of the preparation to create the release package. A list of activities to consider includes:

• Ensure the proper approvals have been documented and received, including:

• Software assurance - "Software assurance shall provide objective evidence to the project and NASA Safety and Mission Assurance (SMA) of the software's readiness for operational release." ²⁷⁸
• Release authority – Change Control Board (CCB) or other authorized "owner."

• Ensure any required acceptance data package has been prepared (see NASA-GB-8719.13, NASA Software Safety Guidebook, ²⁷⁶ for typical content information), including the Version Description Document (5.16 - VDD - Version Description Document).
• Ensure all required configuration audits have been completed (SWE-084 - Configuration Audits).
• Ensure all approved deviations and waivers are documented.
• Ensure all change requests have been completed and verified.
• Ensure all documents and training materials are complete, including installation and any special installation needs/support, customization and configuration documents, user and operator guides, and release notes.
• Ensure all legal issues, such as licensing or export regulations (e.g., International Traffic in Arms Regulations (ITAR)), are addressed, as applicable.
• Ensure any "ready to ship" reviews are completed.
• Ensure all installation sites are prepared to receive and install the release, conducting any pre-installation visits as appropriate.
• Ensure required support personnel is trained and ready to address issues related to the installed release.

See also SWE-083 - Status Accounting, SWE-081 - Identify Software CM Items. See also Topic 5.01 - CR-PR - Software Change Request - Problem Report, 5.06 - SCMP - Software Configuration Management Plan, 

3.2.2 Creation and Delivery of the Release Package

Procedures for creating the release package are used once the preparation steps have been completed and it is time to create the release package. Typically, there is a master copy of the release package, and copies are distributed to customers. Depending on Center policy, the master may be created by the CM group and the copies created, packaged, and shipped by another group. Whatever process is used needs to be clearly defined in the release management procedures.

When developing those procedures, consider the following:

• Identify the scope of the release, including the full set of configuration items (CIs) that are to be included, their versions, and revisions.
• Identify the tools to be used to create the release, including compilers and linkers.
• Identify the software to be used to create the release, including the operating system, macros, and libraries.
• Identify software and tool options to be used (compiler options, environmental parameters).
• Identify the procedures for creating the master copy of the release or reference them if captured elsewhere.
• Identify who generates the master copy of the release package.
• Document the format, layout, and media for the master.
• Document the verification process to confirm the master contains the proper CI's.
• Identify the media to be used for the delivery of copies.
• Document replication procedures to be used to generate copies of the master.
• Document verification procedures to be used to confirm the copies match the master (keep in mind that compilers can insert dates and times, so byte-by-byte compares need to take this into account).
• Document any virus checks that need to be run on the copies before delivery to the customer.
• Document any testing to be performed at the customer site (e.g., regression testing).

When developing procedures for delivery of the created package, consider the following:

• Document whether the release is full, the partial release which requires a previous full release to be installed first, or a patch; if all types will be used, procedures for creating and installing each need to be created.
• Document delivery methods and procedures, including shipping methods, if required.
• Document security measures to be used when handling and shipping the release.
• Determine an installation schedule that works with the customer's schedule.
• Document responsibilities for performing installation and installation testing.
• Document responsibilities for configuring and/or customizing the installed software.
• Document plans to revert to an earlier release of the software, as applicable.

3.2.3 Storage and Maintenance of the Release Package

As part of release management, the master needs to be safely and securely stored following documented procedures. When developing procedures for storing and maintaining the release package, consider the following:

• Document the retention period; e.g., "master copies of all configuration items in a release and the release itself shall be maintained for the life of the product" (IEEE Std 828-2012, IEEE Standard for Configuration Management in Systems and Software Engineering  ²¹⁶)
• Document how to place the master into the CM system with its unique identifier.
• Document access restrictions.
• Document or reference any specific procedures for storing code and documentation for safety or security-critical functions.
• Identify release records, such as the Version Description Document (VDD), to be captured and stored with the release, as applicable.

When defining activities for release management, consider coordinating or applying the same concepts as part of data management activities.  A basic description of data management is provided in SWE-079 - Develop CM Plan.

3.3 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

3.4 Center Process Asset Libraries

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only). 

4. Small Projects

Projects with limited budgets may choose to follow a common Center or project-level release management procedure rather than have separate procedures for each project. Slight modifications may be required for each project, but the overall master process would not have to be developed or maintained on a per-project basis.

5. Resources

5.1 References

5.2 Tools

6. Lessons Learned

6.1 NASA Lessons Learned

The NASA Lesson Learned database contains the following lesson learned related to release management:

• Computer Hardware-Software/International Space Station/Software Configuration Management. Lesson No: 1130 ⁵⁴¹: Although it was "grandfathered" out of NPR 7150.2 compliance, the experience regarding source code for the International Space Station (ISS) is an important caveat when establishing software supplier agreements. NASA does not have source code access for all partners' deliveries for the ISS. The partners cite their concerns that the delivery of source code could compromise their contractors' proprietary data. The ISS has initiated discussions with all partners to reach an agreement on what level of source code visibility is necessary to ensure adequate knowledge by the control centers for on-orbit anomaly resolution. It is not clear how much extra effort these discussions have taken.
• Place Flight Scripts Under Configuration Management Prior to ORT (Project attention to configuration control). Lesson Number 2476 ⁵⁷⁴:   "When flight scripts developed independently by project personnel are not placed under configuration control early enough in-flight software development, multiple versions of the scripts tend to proliferate and cause confusion and delays. The MER, Juno, and GRAIL projects demonstrated the utility of placing flight scripts under change control prior to ORT."
• COTS Change Processing. Lesson Number 3457 ⁵⁸⁰: “The time-sensitive nature of commercial-off-the-shelf (COTS) configuration changes such as the application of routine security patches and vendor updates/upgrades could not be effectively managed using the time-consuming configuration management practices typical of a custom environment. Systems that use both COTS and custom hardware and software products must adopt configuration management practices that effectively address the unique requirements of both product categories."
• Software Design for Maintainability. Lesson Number 0838 ⁵²⁶: Impact of Non-Practice: "Because of increases in the size and complexity of software products, software maintenance tasks have become increasingly more difficult. Software maintenance should not be a design afterthought; it should be possible for software maintainers to enhance the product without tearing down and rebuilding the majority of code."
• International Space Station (ISS)/Multiple Element Integrated Testing (MEIT)/Software Configuration Management.  Lesson Number 1165 ⁵⁸⁵ : “Due to the rapid pace of ISS assembly launches and the many and varied resulting configurations, MultiElement Integration Testing MEIT with operational loads of Portable Computer System PCS software is limited and, in some cases, may only be accomplished in the brief time allocated for regression testing.”
• Redundant Verification of Critical Command Timing (1995).  Lesson Number 559 ⁵⁸⁶: “When a new mission software release was uploaded to the spacecraft, the inflight upload failed to include a software patch that had been written to fix a defective countdown timer. Because an independent watchdog timer was planned but never implemented due to constrained project resources, the thrusters continued to fire after the desired shutdown time and the mission was terminated. Recommendations centered on the need for rigorous software configuration management, a watchdog timer to terminate operations, and testbed verification of in-flight software updates.”
• Reuse of Analysis Software. Lesson Number 2158 ⁵⁸⁷: “When a project plans to reuse existing metrology software for a new application, a thorough independent review of the software and its new interfaces should be conducted.”
• Institutional Configuration Management Organization.  Lesson Number 4516 ⁵⁸⁸: “Configuration and Data Management CDM is performed at the Kennedy Space Center KSC for hardware and software by the different programs and projects independently of the Center/Institution. What is important for CDM is having consistency in policy, guidelines, processes, and tools to provide seamless and efficient transaction of configuration information. A single institutional CDM organization can provide consistent guidance and direction to comply with agency, program, and project CDM requirements.”

6.2 Other Lessons Learned

No other Lessons Learned have currently been identified for this requirement.

7. Software Assurance

7.1 Tasking for Software Assurance

7.2 Software Assurance Products

• Software Configuration Management Baseline and Process/Procedure Audit Report
• Results and findings from audits on CM process performance, including any risks or issues.

  
  

  Objective Evidence

  • Software Problem reporting or defect tracking data
  • Software configuration management system data
  • Software assurance audit results of the configuration management processes.
  • Software data file location data
  • Software version description document(s).
   Definition of objective evidence

  Objective evidence is an unbiased, documented fact showing that an activity was confirmed or performed by the software assurance/safety person(s). The evidence for confirmation of the activity can take any number of different forms, depending on the activity in the task. Examples are:

  • Observations, findings, issues, risks found by the SA/safety person and may be expressed in an audit or checklist record, email, memo or entry into a tracking system (e.g. Risk Log).
  • Meeting minutes with attendance lists or SA meeting notes or assessments of the activities and recorded in the project repository.
  • Status report, email or memo containing statements that confirmation has been performed with date (a checklist of confirmations could be used to record when each confirmation has been done!).
  • Signatures on SA reviewed or witnessed products or activities, or
  • Status report, email or memo containing a short summary of information gained by performing the activity. Some examples of using a “short summary” as objective evidence of a confirmation are:
    • To confirm that: “IV&V Program Execution exists”, the summary might be: IV&V Plan is in draft state. It is expected to be complete by (some date).
    • To confirm that: “Traceability between software requirements and hazards with SW contributions exists”, the summary might be x% of the hazards with software contributions are traced to the requirements.
  • The specific products listed in the Introduction of 8.16 are also objective evidence as well as the examples listed above.

7.3 Metrics

• # of Compliance Audits planned vs. # of Compliance Audits performed.
• # of software process Non-Conformances by life cycle phase over time.
• # of Non-Conformances identified in release documentation (Open, Closed).
• # of process Non-Conformances (e.g., activities not performed) identified by SA vs. # accepted by the project.
• Trends of # Open vs. # Closed over time.
• # of Non-Conformances per audit (including findings from the process and compliance audits, and process maturity).
• # of Open vs. Closed Audit Non-Conformances over time,
• Trends of # of Non-Conformances from audits over time (Include counts from process and standards audits and work product audits.).

See also Topic 8.18 - SA Suggested Metrics. 

7.4 Guidance

For Task 1, software assurance will review project documentation and confirm that procedures are established for storing, handling, release, delivery, and maintenance. These processes and procedures may be found in some of the following documents including the software development/management plan, data management plans, configuration management plan, and maintenance plans or they may reference other project documents or Center document describing their processes. If Center procedures are referenced, they should be tailored for the project-specific details. Any project-established procedures should be checked for compliance with the requirements NPR 7150.2 ⁰⁸³ and any Center requirements.

For Task 2, perform various types of audits to confirm that the procedures reviewed in Task 1 are being followed. These audits would include process/procedure audits to ensure that the activities specified in the procedures are being done. Other audits should include audits done before delivery such as the Functional Configuration Audit (FCA) and Physical Configuration Audit (PCA). The process for configuration management should be audited periodically to ensure the project data is being handled and stored properly. For more information on planning and executing audits, see 8.12 - Basics of Software Auditing.  For more information FCAs and PCAs. see SWE-084 - Configuration Audits.

In addition to these two tasks, for releases and delivery, software assurance will sign off on the delivery documentation such as the FCA and PCA audits to indicate that all required activities have been completed and the required conditions have been met. Some of the activities and conditions that need to be considered are below:

• The software is accepted. Generally, the software is considered when it has met the acceptance criteria as defined in the test plan, satisfies the technical requirements, and meets the requirements in NPR 7150.2, NASA-STD-8739.8 ²⁷⁸, and any Center-level requirements, and any additional contract, MOA, or MOU software requirements. Software acceptance can be rolled into the system acceptance as long as the software has all been tested and signed off by software assurance as concurrence indicates the software meets the requirements for SW quality, SW safety, SW reliability, SW security, and SW V&V, and SW IV&V.
• Software assurance has evaluated and concurred that Project/Provider has satisfactorily completed SW and system verification and validation plans and procedures and that test results are acceptable for the target environment and the intended software use.
• Software assurance has verified SW deliveries by conducting or participating in (and signing off on) the execution of configuration audits, such as Functional Configuration Audit (FCA) and Physical Configuration Audit (PCA). Software Assurance accepts or rejects SW deliveries based on the state of the SW resolution of DRs/PRs and any outstanding risks and satisfactory completion of any safety and reliability verifications. 
• Software assurance has verified the integrity of SW installations against the delivery documentation, data checks, operational set-up, and configuration managed versions. 
• Software assurance has assured that the operations and maintenance environment is prepared for software delivery, installation, operations, and maintenance including the transfer or acquisition of resources (e.g., licenses, test benches, simulators) necessary to conduct operations and maintenance. 

• Software assurance confirms that any operations or maintenance documents are complete, usable, and delivered, including such information as licenses, simulators, models, COTS, open-source, tools, or reuse code incorporated or used, as applicable. 

Software assurance will do the following activities and deliverables for the operations, maintenance, and retirement of the software:

• Assure that the processes for maintenance and retirement are being followed as per their plans
• Assure that the plans for maintenance, operations, and retirement are complete.
• Review changes approved for implementation during maintenance and confirm that:
  • Changes will not impact Project risk posture concerning quality, safety, reliability, security, or NPR 7150.2 compliance
  • Changes are being tested thoroughly, including regression testing
  • Approved changes are documented and reflected in “as-built” documentation
  • Document and raise to the Project any issues, risks, or concerns
• Confirm that retirement is carried out as per the retirement plan and that:
  • All software and documentation are archived
  • Licenses are transferred or canceled
  • Any equipment or furniture in associated labs is properly transferred or disposed of
  • Report on any issues, concerns

Every task that involves performing an audit should also clarify that all audit findings are promptly shared with the project and will be addressed in the handbook guidance.

7.5 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook: