bannera

Book A.
Introduction

Book B.
7150 Requirements Guidance

Book C.
Topics

Tools,
References, & Terms

SPAN
(NASA Only)

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Role

Responsibility

Project Manager

Approve procurement plan

Software Project Lead

Prepare procurement plan, prepare SOW software requirements and software data requirements for the contract, monitor execution of contract

System Engineer

Conduct trade studies, engineering analyses

Contracting Officer (CO)

Prepare contracts

Contracting Officer Technical Representative (COTR)

Prepare contracts

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="30073920ed658213-01937626-41ab4c55-954c8eee-91509be7b9f96e18efbe1119"><ac:plain-text-body><![CDATA[

Software Technical Authority

Prior to contract release verify that the SOW includes the complete flow down of the agency and center software requirements [TOOL:recommended practice]

]]></ac:plain-text-body></ac:structured-macro>

...

2. Ensure proper review of SOW before delivery to procurement/contracts official:

  • Technical Authority to ensure proper flow down of NPR 7150.2A requirements
  • Coordinate with the Safety and Mission Assurance Office to ensure all QA requirements, clauses, and intended delegations are identified and included

3. Identify potential suppliers.

4. Distribute solicitation package

5. Evaluate proposals (typically an evaluation team), based on selection criteria, including:

  • Cost estimation comparisons
  • Evaluation of how well proposed solutions meet the requirements (including interface and technology requirements, NPR 7150.2A requirements)
  • Staff available
  • Past performance
  • Software engineering and management capabilities
  • Prior expertise on similar projects
  • Available resources (facilities, hardware, software, training, etc.)
  • CMMI ratings
    • Check the SEI Published Appraisal Results (PARs) to confirm non-expired rating (http://sas.sei.cmu.edu/pars)
    • Be sure to check the scope of the organization holding the CMMI rating to confirm the rating is held by the specific organization submitting the proposal

6. Select supplier/contractor and document basis for selection.

7. Negotiate and finalize contract:

  • Based on SOW
  • Identify and include management reviews and meetings, such as:
    • Formal reviews, such as those found in NPR 7123.1
    • Technical reviews
    • Progress reviews
    • Peer reviews (see Software Peer Reviews and Inspection topic guidance in this handbook)
    • Software quality assurance meetings
    • System integration test and verification meetings
    • System safety meetings
    • Configuration management meetings
    • Other relevant review for this project
  • Consider for inclusion in contract provisions (description of the method to be used) for verification of
    • Contractor handling of requirements changes
    • Accuracy of contractor transformation of high-level requirements into software requirements and detailed designs
    • Interface specifications between the contractor's product and systems external to it
    • Adequacy of contractor's risk management plan and its implementation in accordance with the required activities in the project Software Risk Management Plan
    • Adequacy of the contractor's integration and test plan and its implementation in accordance with the required activities in the project Software Integration and Test Plan
    • Adequacy of the contractor's configuration management plan and its implementation in accordance with the required activities in the project Software Configuration Management Plan
  • Consider for inclusion in the contract the content and frequency of progress reports and metrics submissions
  • Consider for inclusion in the contract identification of quality records to be maintained by the supplier
  • Consider for inclusion in the contract the delivery process and how it will be accomplished; if incremental development and delivery agreed upon, state how the validation process works (e.g., incremental validation) and whether it requires integration and test with software/hardware products developed by acquirer and/or other contractors or organizations (other institutes, universities, etc.)
  • Consider for inclusion in the contract a policy for maintaining the software after delivery: who is responsible for maintenance of the software, tools, testbeds, and documentation updates

Monitoring and Quality Assurance

Once the provider has been chosen, the acquisition process moves into a monitoring role. The following guidance should be included when establishing the process for provider monitoring and quality assurance:

1. Provide technical requirements interpretation for contractor.

2. Ensure contractor requirements documents meet original intent.

3. Evaluate contractor progress with respect to cost.

4. Periodically monitor contractor skill mix to ensure agreed-upon skills and experience levels are being provided.

5. Oversee government-furnished equipment (GFE) Hover overrelated to the development of software to ensure equipment and information provided in timely manner

6. Periodically assess contractor processes to ensure conformance to process requirements stated in the contract

7. Review and assess adequacy of contractor-provided documentation and ensure contractor implementation of feedback; consider using Formal Inspections to accomplish this task. Track status considering the following example questions:

  • Is the contractor meeting their staffing plan
  • Have the project and the contractor met the user's need?
  • Does the contractor have stable, educated staff?
  • Does the contractor's project have adequate resources (e.g., adequate staffing and computer resources)?
  • Is there realistic planning/budgeting in place?
  • Is the build plan being met?
  • Does the contractor have a good understanding of what is required?
  • Are the requirements stable?
  • Is the completion of designed functionality visible?
  • Is the evolving capability and performance of the contractor's product likely to impact development on the acquirer side of the interface?
  • Are integration and testing proceeding as planned?
  • Is contractor cost/schedule performance on target?
  • Is contractor developing a quality product?

8. Provide regular status reviews to higher-level management on contractor progress.

...

10. Software engineering should provide technical review to the level required to enhance the probability of mission success. (See the Useful Tools section below for a list of areas to consider for software engineering technical review). 

Contract Administration

In addition to monitoring the selection provider's progress and quality, contract administration activities are also carried out for the project. The following guidance should be included when establishing the process for contract administration:

...

  • Typically, an acceptance test plan is created addressing the following:
    • Acquirer and contractor roles and responsibility
    • Defined Test Strategy
    • Defined Test Objectives
    • Defined Acceptance Criteria
    • Developed Test Scenarios
    • Developed Test Scripts
    • Developed Test Matrix
    • Time and Resources Estimate
    • Approval Cycle
    • Strategy for post-delivery problem resolutions
  • Once approved, the test plan is executed and results are documented
    • Select Test Tools
    • Select and Train Team Members
    • Execute the Test Plan (Manual and Automated Methods)
    • Track Test Progress
    • Regression Test
    • Document Test Results
    • Resolve Problem Problem 
    •  

 2. Place formal deliverables under configuration control.

...

For class A software:
The contractor responsible for the acquisition, development or maintenance of class A software shall have a non-expired Capability Maturity Model Integrated? for Development (CMMI-DEV) rating, as measured by a Software Engineering Institute (SEI) authorized or certified lead appraiser, of CMMI DEV maturity level 3 Rating or higher for software, or CMMI-DEV Capability level 3 rating or higher in all CMMI-DEV Maturity Level 2 and 3 process areas for software.

For class B software:
The contractor responsible for the acquisition, development or maintenance of class B software shall have a non-expired Capability Maturity Model Integrated ? for Development (CMMI-DEV) rating, as measured by a Software Engineering Institute (SEI) authorizedDoes SEI use both "authorize" and "certified" with respect to Lead Appraisers in the new v1.3. If not, make consistent. or certified lead appraiser, of CMMI DEV maturity level 2 Rating or higher for software, or CMMI-DEV Capability level 2 rating or higher for software in the following process areas:

...

For class C software, the project can minimally choose to pass down this requirement in accordance with their center's procedures related to class C as long as they provide some action that determines the contractor's capability to develop software in a disciplined manner. As many NASA contractors are already at CMMI ML2 or higher, projects may alternatively choose to simply require ML2 in their RFP for Class C software.

If a contractor chooses to subcontract the development of class A, B, or C software, then the subcontractor is required to have a CMMI ML2 (for class B), ML3 (for class A), or center-specified (for class C) rating in the Supplier Agreement Management process area and levy the project-related requirements on the subcontractor.

...

Example 1:
Information Technology (IT) Management

  • For IT applications, other than mission-specific flight and non-flight software, the Contractor shall use Commercial-off-the-Shelf and existing Government-off-the-Shelf products where cost effective to NASA. All IT applications, other than mission-specific flight and non-mission flight software shall comply with NASA requirements as outlined in NPR 7150.2A, NASA Software Engineering Requirements for the appropriate software classes, limited to classes E, F, and G, and as applicable for the project, MPR 2800.4, and NPR 2830.1 NASA Enterprise Architecture Procedures.


Example 2:

Information Technology Management

  • For IT applications, other than mission-specific software, the Contractor shall:
    • Where cost effective to NASA, use Commercial-Off-The-Shelf (COTS) and existing Government Off-The-Shelf (GOTS) products.
    • Ensure compatibility with existing NASA applications and systems.
    • Comply with NASA requirements for NPR 7150.2A, NASA Software Engineering Requirements for the appropriate software classes, limited to classes E, F, and G, and as applicable for the project.

Examples for RFP Software section

Example 1:
Embedded Software (Firmware)

The Contractor shall develop and maintain software in accordance with NPR 7150.2A, NASA Software Engineering Requirements for the appropriate software classes and as applicable for the project; NASA-STD-8739.8, and NASA Software Assurance Standard (chapters 6 and 7).

...

  1. The Contractor shall define, design, develop, test, qualify, integrate, verify, validate, deliver, and maintain all software. The plans for accomplishing this work shall be documented in DRD_, Software Development Plan_.
  2. The Contractor shall justify the reuse of existing software, modification of existing software, and the development of new software in DRD_, Software Development Plan_.
  3. The Contractor shall, under Project direction, participate in coordinating with the NASA IV&V Facility in accordance with NASA-STD-8739.8, NASA Software Assurance (Chapter 6 and 7) to plan for the participation of the NASA IV&V Facility in the software development lifecycle activities.
  4. The Contractor and its subcontractors' organizations associated with software development responsibilities shall be at Software Engineering Institute Software Capability Maturity Model Integration CMMI --- DEV Maturity Level 3 (Staged Representation) or higher prior to the Preliminary Design Review. This requirement does not apply to commercial-off-the-shelf software procured for the Project.
  5. The Contractor shall develop, update, and maintain all software and software development tools under configuration management in accordance with the DRD_, Software Configuration Management Plan_.
  6. The Contractor shall develop and maintain electronic Software Development Folders for all flight, ground, and test software per DRD_, Software Development Folder_.
  7. The Contractor shall use the following guidance document(s) for the development of all software document deliverables:
    • Project Classification Matrix (use as guidance in interpreting flight software classification definitions in NPR 7150.2A)
  8. The Contractor shall use the following standards for designing, developing, and testing all software:
    • NPR 7150.2A NASA Software Engineering Requirements
    • NASA-STD-8739.8, NASA Software Assurance Standard (chapters 6 and 7)

...

Example 3:
The contractor shall define, design, code, test, integrate, and qualify the software. The contractor shall treat the software component of firmware, which consists of computer programs and data loaded into a class of memory that cannot be dynamically modified by the computer during processing, as software for the purposes of this statement of work. The scope of this activity applies to the re-use of existing software, modification of existing software, and/or development of new software. The contractor shall provide information and access to products under development to provide the government with insight into software development and test activities, including monitoring integration and verification adequacy, auditing the software development process and participation in all software and system reviews. The contractor shall support the implementation of the overall risk management process as well as program status and progress reviews for the software development process. The contractor shall support software Technical Interchange Meetings and other status meetings, as required, to facilitate government insight into the software development. The government insight may include government civil servant insight, government support contractor insight and independent verification and validation review. The contractor shall perform Peer Reviews on Software Requirements Specifications, Software Test Plans, and on selected design and code items and provide results to the government via Software Inspection/Peer Review Reports. The contractor shall maintain software metrics and provide Software Metrics Reports in accordance with DRD.

The contractor shall provide the government web-based electronic access (with access control) to intermediate and final software products (including code) and software process tracking information including software development and management metrics.
The software development shall comply with NPR 7150.2A NASA Software Engineering Requirements as applicable for the project.

Examples for RFP EGSE Software section

Example 1:
EGSE Software

  1. The Contractor shall perform the design, code, Verification, Validation, and delivery of all EGSE code and executables in response to EGSE Subsystem Requirements Document."
  2. The Contractor shall develop and maintain software in accordance with NPR 7150.2A, NASA Software Engineering Requirements for the appropriate software classes and as applicable for the project; and NASA-STD-8739.8, NASA Software Assurance Standard (chapters 6 and 7).

...