1.2.3 The NASA Headquarters' Chief Engineer shall periodically benchmark each Center's software engineering capability against its Center Software Engineering Improvement Plan.
Note: Center Software Engineering Improvement Plans are documented per Center Software Engineering Improvement Plan requirements. Capability Maturity Model Integration (CMMI) for Development (CMMI-DEV) appraisals are the preferred benchmarks for objectively measuring progress toward software engineering process improvement at NASA Centers.
1.2 Applicability Across Classes
The Headquarters Office of the Chief Engineer (OCE) is responsible for ensuring that the Agency-level software engineering requirements and policies are being followed throughout the Agency.
The Headquarters Office of the Chief Engineer (OCE) achieves this requirement by a number of methods:
- OCE Center Surveys. OCE personnel conduct periodic assessment of compliance at the Centers and within the programs/projects to verify that they are meeting this responsibility.
- Review of the CMMI (Capability Maturity Model Integration) appraisal results.
- Review and participation on program and project reviews.
- Review of Center and project planning documents, schedule, and progress.
- Review of Center and project waivers.
- Feedback and status presentations provided by the Centers during the NASA Software Working Group activities.
- Yearly task agreements between the Centers and the NASA Headquarters OCE.
- Feedback and discussion for the NASA Software Working Group members and Mission Software Steering Committee members.
- Project status and feedback provided to the NASA Headquarters OCE.
- Software inventory data.
- External Agency inquires.
The Headquarters OCE performs Center and organizational surveys. These surveys are used by the OCE to provide oversight, to maintain internal control, and to review its operations and assess compliance with Agency policy. The OCE appraisal process addresses several objectives. They are:
- Review Center and specified NASA Headquarters organizations' processes and infrastructure for compliance with OCE requirements and policies.
- Review specific program/project "files" for compliance with requirements and policies.
- Identify systemic problems or deficiencies.
- Recognize areas of excellence/best practices.
- Receive Center feedback regarding areas where Agency policy and requirements may need to be modified.
Currently, the OCE software surveys focus on the following core elements:
- Compliance to Agency-level software engineering requirements and policies.
- Software Engineering technical authority implementation at a Center.
- Software engineering dissenting opinions and waiver process at a Center.
- Software engineering management.
- Incorporation of software engineering lessons learned.
- Software Documentation and Records.
- Software Classification process and levels.
- Software Engineering Training approach.
- Software Risk Management approach.
- Software Cost Estimates and Resources Allocation.
- Software insight responsibilities.
- Use of software engineering discipline for software associated with Programmable Logic Devices.
- Software Architecture and Detail Design.
- Software Safety and assurance practices.
- Software Inventory data.
- Inter-Center Software Work.
- Use and recording of software metric data.
The CMMI-DEV benchmarking activities will evaluate the Center's current and, with follow-on evaluations, improved capabilities in the specific and general practices of software engineering. The CMMI requirement is a qualifying requirement. The requirement is included to make sure NASA projects are supported by software development organization(s) having the necessary skills and processes in place to produce reliable products within cost and schedule estimates. This requirement provides NASA with a methodology to measure software development organizations against an industry-wide set of best practices that address software development and maintenance activities applied to products and services. The CMMI is a yardstick against which the maturity of an organization's product development and acquisition processes can be measured and compared with industry state of the practice. The CMMI requirement provides NASA with an industry standard approach to help measure and ensure compliance with the intent of the NPR 7150.2 process related requirements. This requirement provides NASA with a common methodology to assess internal and external software development organizations processes and helps NASA identify potential risk areas within a given organization's software development processes. See SWE-032 for rating requirements and the CMMI material on the Software Engineering Institute's website, which describes the current CMMI model that is used in the evaluation of a Center's software development capabilities.
Additional guidance related to OCE Benchmarking of the Center Plan development and progress may be found in the following related requirements in this handbook:
Center Improvement Plans
CMMI Levels for Class A, B, and C Software
Software Process Determination
Center SW Improvement Plan
4. Small Projects
Typically the Office of the Chief Engineer (OCE) includes a small project in the survey activities at a Center. The OCE survey leader will work with the Center SW POC (Point of Contact) to develop the appropriate level of survey involvement for small projects.
6. Lessons Learned
No Lessons Learned have currently been identified for this requirement.