bannera

Book A.
Introduction

Book B.
7150 Requirements Guidance

Book C.
Topics

Tools,
References, & Terms

SPAN
(NASA Only)

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 20 Next »

Error formatting macro: alias: java.lang.NullPointerException
SWE-032 - CMMI levels for Class A, B, and C Software
Unknown macro: {div3}

1. Requirements

2.5.1. The project shall ensure that software is acquired, developed, and maintained by an organization with a non-expired Capability Maturity Model Integration® for Development (CMMI-DEV) rating  as measured by a Software Engineering Institute (SEI) authorized or certified lead appraiser as follows:

For Class A software:

CMMI-DEV Maturity Level 3 Rating or higher for software, or CMMI-DEV Capability Level 3 Rating or higher in all CMMI-DEV Maturity Level 2 and 3 process areas for software.

For Class B software:

CMMI-DEV Maturity Level 2 Rating or higher for software, or CMMI-DEV Capability Level 2 Rating or higher for software in the following process areas:

         a. Requirements Management.
         b. Configuration Management.
         c. Process and Product Quality Assurance.
         d. Measurement and Analysis.
         e. Project Planning.
         f. Project Monitoring and Control.
         g. Supplier Agreement Management (if applicable).

For Class C software:

The required CMMI-DEV Maturity Level for Class C software will be defined per Center or project requirements.

1.1 Notes">1.1 Notes

Organizations who have completed Standard CMMI® Appraisal Method for Process Improvement (SCAMPISM) Class A appraisals against the CMMI-DEV model are to maintain their rating and have their results posted on the SEI Web site so that NASA can assess the current maturity/capability rating.  Software development organizations need to be reappraised and keep an active appraisal rating posted on the SEI Web site during the time that they are responsible for the development and maintenance of the software.

For Class A software development only, a transition period to obtain a CMMI-DEV Maturity/Capability Level 3 Rating will be allowed for organizations developing Class A software per the NASA Headquarters' Office of the Chief Engineer's approved Center Software Engineering Improvement Plan as described in SWE-003, SWE-004, and SWE-108.

For Class B software, in lieu of a CMMI rating by a development organization, the project will conduct an evaluation, performed by a qualified evaluator selected by the Center Engineering Technical Authority, of the seven process areas listed in SWE-032 and mitigate any risk, if deficient.  This exception is intended to be used in those cases in which NASA wishes to purchase a product from the "best of class provider," but the best of class provider does not have the required CMMI rating.  When this exception is exercised, the Center Engineering Technical Authority should be notified.

Implementation Notes from Appendix D include the following additional information:

(EHB Editor note: Class B entry includes "Note 1", which duplicates the text provided in the previous section.)

1.2 Applicability Across Classes

Class C is labeled with "P (Center)".  This means that an approved Center-defined process that meets a non-empty subset of the full requirement can be used to achieve this requirement. "Note 1" can also be applied to the implementation of Class C software for this requirement.

Class

  A_SC 

A_NSC

  B_SC 

B_NSC

  C_SC 

C_NSC

  D_SC 

D_NSC

  E_SC 

E_NSC

     F      

     G      

     H      

Applicable?

   

   

    X

    X

    P(C)

    P(C)

   

   

   

   

   

   

   

Key:    A_SC = Class A Software, Safety Critical | A_NSC = Class A Software, Not Safety Critical | ... | - Applicable | - Not Applicable
X - Applicable with details, read above for more | P(C) - P(Center), follow center requirements or procedures

Unknown macro: {div3}

2. Rationale

The CMMI requirement is a qualifying requirement.  The requirement is included to make sure NASA projects are supported by software development organization(s) having the necessary skills and processes in place to produce reliable products within cost and schedule estimates.

This requirement provides NASA with a methodology to:

  • Measure software development organizations against an industry-wide set of best practices that address software development and maintenance activities applied to products and services.
  • Measure and compare the maturity of an organization's product development and acquisition processes with industry state of the practice.
  • Measure and ensure compliance with the intent of the NPR 7150.2 process related requirements using an industry standard approach 
  • Assess internal and external software development organizations processes
  • Identify potential risk areas within a given organization's software development processes

The Capability Maturity Model (CMM) and the Capability Maturity Model Integration for Development (CMMI-DEV) is an internationally used framework for process improvement in development organizations. It is an organized collection of best practices and proven processes areas that thousands of software organizations have used and been appraised against for over the past two decades. CMMI defines practices that businesses have implemented on their way to success.

Practices cover topics that include eliciting and managing requirements, decision making, measuring performance, planning work, handling risks, and more. Using these practices, NASA can improve NASA software projects' chances of mission success. CMMI ratings can cover a team, a work group, a project, a division, or an entire organization. When evaluating software suppliers, it's important to make sure that the specific organization doing the software work on the project has the cited rating (as some parts of a company may be rated while others are not).

Benefits of using CMMI include:

  • Reducing risk of software failure - Increasing mission safety,
  • Improving the accuracy of schedule and cost estimates by requiring the use of historical data and repeatable methods
  • Helping NASA become a smarter buyer of contracted out software,
  • Increasing quality by finding and removing more defects earlier,
  • Improving the potential for reuse of tools and products across multiple projects,
  • Increasing ability to meet the challenges of evolving software technology,
  • Improving Software development planning across the Agency,
  • Improving NASA contractor community with respect to software engineering,  
  • Lowering the software development cost,
  • Improving employee morale,
  • Improving customer satisfaction,
  • Improving NASA and Contractor community knowledge and skills,
  • Providing NASA a solid foundation and structure for developing software in a disciplined manner.

The first Software CMM* Level 5 project was the supplier and maintainer of flight software for NASA's Space Transport System (Shuttle) in 1989. Since NASA's adoption of the CMMI framework for software development, CMMI has been widely used at NASA Centers as well as the Agency's contractor community. Carnegie Mellon University's Software Engineering Institute is the steward of CMMI which was developed under Department of Defense funding.

Unknown macro: {div3}

3. Guidance

It's important to note that SWE-032 and a CMMI-DEV rating is an organizational qualifier to acquire, develop, or maintain software for or by NASA for Classes A, B, & C.

Many of the requirements in NPR 7150.2 are consistent with the established process areas in the CMMI-DEV framework. The CMMI-DEV rating as well as consistent NPR 7150.2 requirements are both needed to ensure that organizations have demonstrated the capability to perform key software engineering processes and have a binding agreement to continue to execute key software engineering processes during the development of NASA's most critical software systems. 

This requirement applies to both Safety Critical as well as Not Safety Critical software in Classes A, B, and C.  Review the status of the software development and or maintenance organization's CMMI rating at each major life cycle review. 

Class A software – if you acquire, develop or maintain Class A software the organization performing the functions is required to have a non-expired CMMI-DEV Level 3 or higher rating. 

Class A software acquisition guidance – When a Project seeks to acquire a system that includes Class A software, the Project's acquisition team is required to have support from personnel in an organization that has been rated at a CMMI Level 3 or higher. Evidence that a CMMI-DEV Level 3 rated organization has participated in the acquisition activities could include direct support on the acquisition team and or review and approval of the acquisition products by the CMMI-DEV rated organization.  The extent of the CMMI-DEV Level 3 rated organizations support required for a Class A acquisition should be determined by the Center's Engineering Technical Authority responsible for the project. 

Identification of the appropriate personnel from a organization that has been rated at a CMMI-DEV Level 3 or higher to support the Project acquisition team is the responsibility of the designated Center Engineering Technical Authority and Center Management.  The Center Engineering Technical Authority has the responsibilities for ensuring that the appropriate and required NASA Software Engineering requirements are included in an acquisition.

For those cases in which a Center or project desires a general exclusion from the NASA Software Engineering requirement(s) in this NPR or desires to generically apply specific alternate requirements that do not meet or exceed the requirements of this NPR, the requester shall submit a waiver for those exclusions or alternate requirements for approval by the NASA Headquarters' Chief Engineer with appropriate justification (see [SWE-120]).

Class A software development or maintenance guidance - The software organizations that directly develop or maintain Class A software are required to have a valid CMMI-DEV Level 3 or higher rating for the organization performing the activities.  Support contracts supporting NASA in-house software development organizations can be included in the NASA organizational assessments.  Project contractors and subcontractors performing Class A software development are required to have their own CMMI-DEV Level 3 rating.  It is important for NASA and primes to pass this requirement down in contracts to ensure all subcontractors have the necessary CMMI-DEV rating.

The CMMI-DEV Level 3 rating is to be maintained throughout the project's development or maintenance period.  NASA requests organizations' CMMI ratings be posted on the SEI website http://sas.sei.cmu.edu/pars/.SEI vets the validity of the CMMI appraisals on this list and assures the rating hasn't expired (as of this writing CMMI ratings are valid for a 3 year period). In rare instances (rating earned in a classified environment) an organization may have a current CMMI-DEV rating, but it doesn't appear on the SEI website. In these cases the supplier's claim can be directly checked with SEI.

Class B software - CMMI-DEV Maturity Level 2 Rating or higher for software, or CMMI-DEV Capability Level 2 Rating or higher for software in the following process areas:

         a. Requirements Management.

         b. Configuration Management.

         c. Process and Product Quality Assurance.

         d. Measurement and Analysis.

         e. Project Planning.

         f. Project Monitoring and Control.

         g. Supplier Agreement Management (if applicable).

Class B software acquisition guidance - In the case of the Project acquiring a system that includes Class B software, the Project's acquisition team is required to have support from organizations that have been rated at CMMI-DEV Level 2 (via a Continuous or Staged approach), including the process area Supplier Agreement Management. Evidence that a CMMI-DEV Level 2 rated organization has participated in the acquisition activities could include direct support on the acquisition team and or review and approval of the acquisition products by the CMMI-DEV rated organization. 

The Center Engineering Technical Authority responsible for the project determines the extent of the CMMI-DEV Level 2 rated organization's support required for a Class B acquisition. Identification of the appropriate personnel from a organization that has been rated at a CMMI-DEV Level 2 or higher to support the Project acquisition team is the responsibility of the designated Center Engineering Technical Authority and Center Management.  The Center Engineering Technical Authority has the responsibilities for ensuring that the appropriate and required NASA Software Engineering requirements are included in an acquisition.

For those cases in which a Center or project desires a general exclusion from the NASA Software Engineering requirement(s) in this NPR or desires to generically apply specific alternate requirements that do not meet or exceed the requirements of this NPR, the requester shall submit a waiver for those exclusions or alternate requirements for approval by the NASA Headquarters' Chief Engineer with appropriate justification (see [SWE-120]).

Class B software development or maintenance guidance - The software organizations that directly develop or maintain Class B software are required to have a valid CMMI-DEV Level 2 or higher rating (via a Continuous or Staged approach) for the organization performing the activities.  Support contracts supporting NASA in-house software development organizations can be included in the NASA organizational assessments.  Project contractors and subcontractor performing Class B software development are required to have their own CMMI-DEV Level 2 or higher rating.  The CMMI-DEV Level 2 maintains an active rating during the development or maintenance period.  The rating is to be posted on the SEI website http://sas.sei.cmu.edu/pars/.

Guidance on the exception for Class B software development and maintenance - If this option is used, the Project is responsible for funding the evaluation and for addressing any all risks that are identified during the evaluation.  A SCAMPI B or SCAMPI C appraisal across the listed process areas in this requirement is one method for conducting this evaluation. The Center Engineering Technical Authority is responsible for maintaining all records associated with the evaluation for the life of the project.  The decision on participators in the evaluation process is determined by the responsible Center Engineering Technical Authority on the project.  Recommended guidance is that the "qualified evaluator" should have demonstrated experience on a SCAMPI A appraisal or training such as CMMI Practitioner Level 2 training, completion of a intro to CMMI training course should not be the only criteria used in the selection.

Class C software -The required CMMI-DEV Maturity Level for Class C software will be defined per Center/project requirements.

Class C software acquisition guidance – Center level directives provide information on how to satisfy " P(Center)" requirements for SWE-032 for Class C software. In the case of the Project acquiring a system that includes Class C software it is recommended that the Project's acquisition team have support from personnel in an organization that has been rated at a CMMI-DEV Level 2, including the process area, Supplier Agreement Management. The Center decides the extent and evidence required that personnel in a CMMI-DEV Level 2 rated organization have participated in the acquisition activities. The extent of the CMMI-DEV Level 2 rated organizations support required for a Class C acquisition is determined by the Center Engineering Technical Authority responsible for the project. If a Center does not have any CMMI-DEV Level 2 rated organizations then the acquisition team can be supported by software engineers knowledgeable of the CMMI-DEV software development practices. 

Identification of the appropriate personnel that are knowledgeable of the CMMI-DEV software development practices to support the Project acquisition team is the responsibility of the designated Center Engineering Technical Authority and Center Management.  The Center Engineering Technical Authority has the responsibilities for ensuring that the appropriate and required NASA Software Engineering requirements are included in an acquisition.

For those cases in which a Center or project desires a general exclusion from the NASA Software Engineering requirement(s) in this NPR or desires to generically apply specific alternate requirements that do not meet or exceed the requirements of this NPR, the requester shall submit a waiver for those exclusions or alternate requirements for approval by the NASA Headquarters' Chief Engineer with appropriate justification (see [SWE-120]).

Class C software development or maintenance guidance – Recommended practice is that all software organizations that directly develop or maintain Class C software have valid CMMI-DEV Level 2 rating for the organization performing the activities.  Support contracts supporting NASA in-house software development organizations can be included in the NASA organizational assessments. 

Project contractors and subcontractors performing Class C software development are required to have their own CMMI-DEV Level/Capability rating consistent with Center/project requirements.  These organizations are to maintain software process areas as well as an active CMMI-DEV rating during the development or maintenance period.  The rating is to be posted on the SEI website http://sas.sei.cmu.edu/pars/

If a Center does not have a CMMI-DEV Level 2 rated organization, then the development and or maintenance team should be supported by a software engineer who has as minimum completed the CMMI Practitioner Level 2 training and is knowledgeable of the CMMI-DEV software development practices. The level of support required is based on project risk and is determined by the Center Technical Authority responsible for the software activities.

Unknown macro: {div3}

4. Small Projects

<ac:macro ac:name="unmigrated-wiki-markup">
<ac:plain-text-body><![CDATA[

NDIA

]]></ac:plain-text-body>
</ac:macro>

CMMI Working Group conducted a study on the use of CMMI-DEV within Small Businesses in 2010 158. One of the counterintuitive findings was that the "Perceptions that CMMI is too burdensome for small businesses is not supported by data on CMMI-DEV adoption". Significant numbers of organizations in the 1-20 employees range adopted and achieved CMMI Level ratings.

Small projects are expected to take advantage of Center or organizational assets. 

Unknown macro: {div3}

5. Resources


5.1 Tools

Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN).

NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.

The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.

Unknown macro: {div3}

6. Lessons Learned

 A documented lesson from the NASA Lessons Learned database notes the following:

Lesson Number 1414: "Since the procurement's goal is to minimize the time from start to finish, part of its philosophy is to instill efficiency into the Contractor-Government roles and relationships. Thus, it becomes paramount during the selection process to ensure that the Contractor's processes, procedures, and tools are adequate (as based on some established criteria such as ISO 9001 and/or CMMI) to allow the Government to take a "hands-off" approach during implementation. Also, any criteria to be used to verify/validate and or assess the Contractor's work after contract award must be consistent and compatible with the performance criteria levied on the Contractor." (http://www.nasa.gov/offices/oce/llis/1414.html )

Additional CMM/CMMI Lessons Learned by NASA associated with implementing and maintaining this requirement are:

  • Preparing for an appraisal helps you get the measurable process improvement
  • CMMI process helped Centers establish a baseline of where they are
  • Organizations develop an extensive set of "tools" (i.e., templates, spreadsheets) to help projects with CMMI practices and artifacts
  • Use of toolset helped projects reach compliance much faster
  • Use of organizations tools help support small project development efforts
  • Software Engineers that have participated in the CMMI process can be mentors that can help implement project tools and help projects utilize and tailor the software development processes
  • The CMMI process helps establish sponsorship across departments and with the Engineering management
  • Establish a relationship early with the CMMI Lead Appraiser
  • PIID development depends on a good artifact collection process and data management approach
  • The CMMI workshops can be used to review the processes in depth and reinforce the tool sets
  • The CMMI process help establish a method of tracking progress on software development activities
  • The CMMI process improves the project management and software configuration management areas
  • CMMI assessments help identify areas for process and project improvement
  • Projects appreciate systematic and analytical feedback on what they are doing
  • Measurement and analysis is a big challenge in the CMMI process
  • Improved quality and review of management plan early in the life cycle and reuse of the plans for new projects
  • Resource planning and tracking at the individual process level provided little additional benefit to the projects
  • Smaller projects need to have light-weight processes to avoid being smothered (especially for a one person task)

As part of its annual review, the Aerospace Advisory Panel included this finding in the Computer Hardware/Software section of its Annual Report for 2000: "NASA has initiated plans to have its critical systems processes evaluated according to the Capability Maturity Model (CMM) of the Software Engineering Institute and to work toward increasing the CMM level of its critical systems processes." (Aerospace Advisory Panel Annual Report for 2000, Finding #17)

  • No labels