bannera

Book A.
Introduction

Book B.
7150 Requirements Guidance

Book C.
Topics

Tools,
References, & Terms

SPAN
(NASA Only)

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migration of unmigrated content due to installation of a new plugin


{alias:SWE-129} {tabsetup:1. The Requirement|2. Rationale|3. Guidance|4. Small Projects|5. Resources|6. Lessons Learned} {div3:id=tabs-1} h1. 1. Requirements
Wiki Markup
Tabsetup
1. The Requirement
1. The Requirement
12. Rationale
23. Guidance
34. Small Projects
45. Resources
56. Lessons Learned


Div
idtabs-1

1. Requirements

6.3.8

The

NASA

Headquarters'

Office

of

the

Chief

Engineer

shall

authorize

appraisals

against

selected

requirements

in

this

NPR

(including

NASA

Headquarters'

Office

of

the

Chief

Engineer

approved

subsets

and

alternative

sets

of

requirements)

to

check

compliance.

h2. {color:#003366}{*}

1.1

Notes{*}{color} NPR

Notes

NPR 7150.2

, NASA Software Engineering Requirements, does

not

include

any

notes

for

this

requirement.

h2.

1.2

Applicability

Across

Classes

{applicable:asc=1\|ansc=1\|bsc=1\|bnsc=1\|csc=1\|cnsc=1\|dsc=1\|dnsc=1\|esc=1\|ensc=1\|f=1\|g=1|h=1|} {div3} {div3:id=tabs-2} h1. 2. Rationale The Headquarters Office of the Chief Engineer (OCE) is responsible for promoting and monitoring software engineering practices throughout the agency. It achieves this in part by administering software requirements, policy, procedures, processes, statutes, and regulations. Headquarters OCE uses continuing periodic oversight of compliance at the Centers and programs/projects to verify that this responsibility is being met. [NPR 7150.2|http://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=7150&s=2] serves as the basis for compliance appraisals for software engineering. The appraisal typically occurs during an OCE survey of a Center's processes and directives, and through examinations of a project's official records. These surveys are one of the tools used by the OCE to provide oversight, to maintain internal control and to review its operations.  While SWE-129 is written from the OCE point of view, the requirement also contains an inherent Center role, i.e., participation in the OCE survey activities. A Center's support of this SWE can be assessed by considering the extent of its preparations for and involvement in these OCE surveys. {div3} {div3:id=tabs-3} h1. 3. Guidance The Headquarters Office of the Chief Engineer (OCE) controls and maintains an appraisal process for use in periodic Center and project [OCE compliance surveys|https://nen.nasa.gov/web/oce/ta].  The OCE compliance survey achieves several objectives. They are: * Review Center and specified NASA Headquarters organizations' processes and infrastructure for compliance with OCE requirements, policy, procedures, processes, statutes, and regulations * Review specific program/project "files" for compliance with requirements, policy, procedures, processes, statutes, and regulations * Identify systemic problems or deficiencies * Recognize areas of excellence/best practices * Receive Center feedback regarding areas where agency policy and requirements should be modified Currently, the OCE compliance surveys focus on the following core elements: * Common framework for unified program and project life cycle * Program and project review structure * Technical authority implementation * Dissenting opinions and deviation/waiver process * Software engineering management * Systems engineering * Lessons learned * Technical standards * Other \\ In addition to [NPR 7150.2|http://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=7150&s=2], the Headquarters OCE compliance survey may also include a review and appraisal of the products resulting from use of the following documents, to the extent they involve software engineering: * NPD 7120.4D, NASA Engineering and Program/Project Policy * NPR 7123.1A, NASA Systems Engineering Requirements * NASA STD-8739.8 Software Assurance Standard * NASA STD-8719.13 Software Safety Standard The two NASA standards in this list are traditionally covered in detail by OSMA audits conducted by the NASA Safety Center. {floatbox} "The baseline set of questions are reviewed and may be revised as needed to support the survey at each specific organization. Input for updates to the questions is obtained from survey team members including the software engineering sub-team lead, the systems engineering sub-team lead, and the OCIO representative for records management" (OCE Requirements Compliance Survey Process, 2010). {floatbox} The OCE and appraisal teams chartered by the OCE plan the scope and content of the survey. A Survey Leader is typically named by the OCE to provide overall event planning and coordination, as well as serving as a liaison between the OCE and the Center's point of contact or survey manager. The process of determining the scope for a survey addresses the following items at a minimum: * Requirements implementation and compliance * Results from audits, reviews, and assessments conducted by other organizations * Trends identified across the Agency or within a single organization Preparations for the survey typically include reviews of the flow down of NASA OCE requirements to Center procedural documents, reviews of organization and program/project specific documentation and reviews of other surveys, audits and assessments. The complete description of this nominally week-long event can be found in the OCE Requirements Compliance Survey Process  file located on the NASA Engineering Network (NEN) website. See the OCE Requirements Compliance Survey Process  document for information about team formation, a generic time-line, and other helpful guidance. What follows in this guidance is a brief summary of the software engineering survey team's appraisal process. The main thrust of the software sub-team's appraisal is built into a set of twenty five questions from the OCE. This baseline set of questions serves as guidance to the Center or project to communicate what the OCE wants to review. The survey leader will communicate these questions to the Center's survey manager three to four weeks before the event, who in turn conveys them to the software point of contact. (SW POC). This is usually the Center's NASA Software Working Group (SWG) primary representative.  The [2010 OCE SW Survey Generic Worksheet|https://nen.nasa.gov/web/software/documents?p_p_id=webconnector_WAR_webconnector_INSTANCE_9BkF&p_p_lifecycle=1&p_p_state=normal&p_p_mode=view&p_p_col_id=column-2&p_p_col_count=1&_webconnector_WAR_webconnector_INSTANCE_9BkF_edu.wisc.my.webproxy.URL=https%3A], located on the NEN website, is a useful template for preparing responses to the specific questions. The set of provided questions are typically the same for all Center surveys, although the actual questions included in the survey may be tailored, based on results learned/obtained from other appraisal activities.  Reviews of partial Center (P (Center)) determinations (see [SWE-140|SWE-140]) will be included in the survey activities. Also included will be reviews of general exclusions or alternate requirements approved against requirements in NPR 7150.2 (see [SWE-120|SWE-120]).  The software appraisal activity begins during the survey planning process with the OCE Pre-Brief Presentation. A major component of this pre-brief is the discussion of selection parameters that are used to determine which projects and software activities will be a part of the survey. This pre-brief meeting is nominally held 6-8 weeks prior to the start of the survey. The actual survey event includes entrance presentations, document reviews by the OCE survey team, interviews of Center and project personnel by the survey team, development and review of initial findings, a review for surfacing general or systemic findings, and a summary presentation to members of management. The last activity is usually an exit briefing, which is the survey team's first formal opportunity to present their findings to the senior management of the surveyed organization.  A series of follow-on actions are defined and assigned. As in many reviews, surveys, and appraisal activities, compliance is often measured against objective evidence.   Expected content and types of this objective quality evidence are usually defined in the preparation for the software survey. This defined material provides the basis for confirmation of compliance with requirements and identification of strengths and weaknesses.  The [Carnegie Mellon University Software Engineering Institute|http://www.sei.cmu.edu/] provides the Capability Maturity Model Integration (CMMI) [appraisal method|http://www.sei.cmu.edu/reports/11hb001.pdf] for process improvement.  The method provides an excellent discussion of the philosophy and types of objective evidence. The degree to which Centers follow the CMMI method can be assessed in a Standard CMMI Appraisal Method for Process Improvement (SCAMPI ) activity. While the information presented in the SCAMPI discussion is centered on evaluating (CMMI) process implementation, the discussions and explanations in the CMMI text provide good background information for use in the OCE appraisals to people for who are relatively inexperienced in appraisals and surveys. {panel} Findings resulting from the survey are generally classified as strengths, weaknesses, observations, opportunities, and non-compliances.  See the Requirements Compliance Survey Process document for definitions of these terms. However, the survey team has a clear and overriding obligation to identify all items of non-compliance and items that adversely affect safety or quality.  These items will be included in the final report.  Significant issues are brought to the immediate attention of the surveyed organization's management via the survey manager {panel} {div3} {div3:id=tabs-4} h1. 4. Small Projects Typically, the Office of the Chief Engineer includes a small project in the survey activities at a Center. The OCE survey leader will work with the Center SW POC to develop the appropriate level of survey involvement for small projects. {div3} {div3:id=tabs-5} h1. 5. Resources These first three files can be found in the OCE Software Survey Instructions and Templates folder, within the OCE NASA Engineering Network (NEN) site. Taken as a group, these documents provide guidance to both headquarters and Center personnel for the preparation and conduct of OCE NPR 7150.2 appraisal surveys. # [OCE Requirements Compliance Survey Process, 2010|https://nen.nasa.gov/web/software/documents?p_p_id=webconnector_WAR_webconnector_INSTANCE_9BkF&p_p_lifecycle=1&p_p_state=normal&p_p_mode=view&p_p_col_id=column-2&p_p_col_count=1&_webconnector_WAR_webconnector_INSTANCE_9BkF_edu.wisc.my.webproxy.URL=https%3A] # [2010 OCE SW Generic Workbook, 2010|https://nen.nasa.gov/web/software/documents?p_p_id=webconnector_WAR_webconnector_INSTANCE_9BkF&p_p_lifecycle=1&p_p_state=normal&p_p_mode=view&p_p_col_id=column-2&p_p_col_count=1&_webconnector_WAR_webconnector_INSTANCE_9BkF_edu.wisc.my.webproxy.URL=https%3A] # [OCE Pre-Brief Presentation Template, 2010|https://nen.nasa.gov/web/software/documents?p_p_id=webconnector_WAR_webconnector_INSTANCE_9BkF&p_p_lifecycle=1&p_p_state=normal&p_p_mode=view&p_p_col_id=column-2&p_p_col_count=1&_webconnector_WAR_webconnector_INSTANCE_9BkF_edu.wisc.my.webproxy.URL=https%3A] # [NASA Systems Engineering Handbook|http://education.ksc.nasa.gov/esmdspacegrant/Documents/NASA%20SP-2007-6105%20Rev%201%20Final%2031Dec2007.pdf], NASA/SP-2007-6105 Rev1, 2007 # [NASA Directives and Charters Procedural Requirements|http://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=1400&s=1E], NPR 1400.1E, 2011 # [NASA Systems Engineering Processes and Requirements|http://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=7123&s=1A] with Change 1, NPR 7123.1A, 2009 # NASA Headquarters Office of the Chief Engineer [engineering deviations and waivers website|https://nen.nasa.gov/web/oce/ta] # [NASA Software Safety Standard|https://standards.nasa.gov/documents/detail/3314914], NASA STD 8719.13  (Rev B w/ Ch1 of 7/8/2004), 2004 # [NASA Software Assurance Standard|https://standards.nasa.gov/documents/detail/3315130], NASA STD 8739.8, 2005 # [NASA Governance and Strategic Management Handbook|http://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPD&c=1000&s=0A], NPD 1000.0A, 2008 # [NASA Engineering and Program/Project Management Policy|http://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPD&c=7120&s=4D], NPD 7120.4D, 2010 # [NASA Space Flight Program and Project Management Requirements|http://nodis3.gsfc.nasa.gov/npg_img/N_PR_7120_005D_/NM_7120-81_.pdf], NPR 7120.5D (NM-7120.81), 2009. # [NASA Software Formal Inspection Standard|https://standards.nasa.gov/documents/detail/3314883], NASA-STD-2202-93, 1993 {toolstable} {div3} {div3:id=tabs-6} h1. 6. Lessons Learned No lessons learned have currently been identified for this requirement. {div3} {tabclose}


applicable
f1
g1
h1
ansc1
asc1
bnsc1
csc1
bsc1
esc1
cnsc1
dnsc1
dsc1
ensc1



Div
idtabs-2

2. Rationale

The Headquarters' Office of the Chief Engineer (OCE) is responsible for promoting and monitoring software engineering practices throughout the agency. It achieves this in part by administering software requirements, policy, procedures, processes, statutes, and regulations. The Headquarters' OCE uses continuing periodic oversight of compliance at the Centers and programs/projects to verify that this responsibility is being met.

NPR 7150.2 serves as the basis for compliance appraisals for software engineering. The appraisal typically occurs during an OCE survey of a Center's processes and directives and through examinations of a project's official records. These surveys are one of the tools used by the OCE to provide oversight, to maintain internal control, and to review its operations. 

While SWE-129 is written from the OCE point of view, the requirement also contains an inherent Center role, i.e., participation in the OCE survey activities. A Center's support of this SWE can be assessed by considering the extent of its preparations for and involvement in these OCE surveys.


Div
idtabs-3

3. Guidance

The Headquarters OCE controls and maintains an appraisal process for use in periodic Center and project OCE compliance surveys.

sweref
374
374
 The OCE compliance survey achieves several objectives. They are:

  • Review Center and specified NASA Headquarters organizations' processes and infrastructure for compliance with OCE requirements, policy, procedures, processes, statutes, and regulations.
  • Review specific program/project "files" for compliance with requirements, policy, procedures, processes, statutes, and regulations.
  • Identify systemic problems or deficiencies.
  • Recognize areas of excellence/best practices.
  • Receive Center feedback regarding modifications in Agency policy and requirements.

Currently, the OCE compliance surveys focus on the following core elements:

  • Common framework for unified program and project life cycle.
  • Program and project review structure.
  • Technical Authority implementation.
  • Dissenting opinions and deviation/waiver process.
  • Software engineering management.
  • Systems engineering.
  • Lessons learned.
  • Technical standards.
  • Other.

In addition to NPR 7150.2, the Headquarters' OCE compliance survey may also include a review and appraisal of the products resulting from use of the following documents, to the extent they involve software engineering:

  • NPD 7120.4D, NASA Engineering and Program/Project Management Policy.
    sweref
    257
    257
  • NPR 7123.1A, NASA Systems Engineering Processes and Requirements.
    sweref
    041
    041
  • NASA-STD-8739.8, Software Assurance Standard.
    sweref
    278
    278
  • NASA-STD-8719.13, Software Safety Standard.
    sweref
    271
    271

The two NASA Standards in this list are traditionally covered in detail by Office of Safety and Mission Assurance (OSMA) audits conducted by the NASA Safety Center.


Floatbox

"The baseline set of questions are reviewed and may be revised as needed to support the survey at each specific organization. Input for updates to the questions is obtained from survey team members including the software engineering sub-team lead, the systems engineering sub-team lead, and the Office of the Chief Information Officer (OCIO)representative for records management" (OCE Requirements Compliance Survey Process, 2010).

sweref
374
374


The OCE and appraisal teams chartered by the OCE plan the scope and content of the survey. A Survey Leader is typically named by the OCE to provide overall event planning and coordination, as well as serving as a liaison between the OCE and the Center's point of contact or survey manager.

The process of determining the scope for a survey addresses the following items at a minimum:

  • Requirements implementation and compliance
  • Results from audits, reviews, and assessments conducted by other organizations
  • Trends identified across the Agency or within a single organization

Preparations for the survey typically include reviews of the flow down of NASA OCE requirements to Center procedural documents, reviews of organization and program/project specific documentation and reviews of other surveys, audits and assessments. The complete description of this nominally week-long event can be found in the OCE Requirements Compliance Survey Process

sweref
374
374
 file located on the NASA Engineering Network (NEN) website.
sweref
258
258
See the OCE Requirements Compliance Survey Process  document
sweref
374
374
for information about team formation, a generic time-line, and other helpful guidance.

What follows in this guidance is a brief summary of the software engineering survey team's appraisal process. The main thrust of the software sub-team's appraisal is built into a set of 25 questions from the OCE. This baseline set of questions serves as guidance to the Center or project to communicate what the OCE wants to review. The survey leader will communicate these questions to the Center's survey manager 3 to 4 weeks before the event, who in turn conveys them to the software point of contact. (SW POC). This is usually the Center's NASA Software Working Group (SWG) primary representative.

 The 2010 OCE SW Survey Generic Worksheet

sweref
352
352
, located on the NEN website, is a useful template for preparing responses to the specific questions.

The set of provided questions are typically the same for all Center surveys, although the actual questions included in the survey may be tailored, based on results learned/obtained from other appraisal activities.  Reviews of partial Center (P (Center)) determinations (see SWE-140) will be included in the survey activities. Also included will be reviews of general exclusions or alternate requirements approved against requirements in NPR 7150.2 (see SWE-120). 

The software appraisal activity begins during the survey planning process with the OCE Pre-Brief Presentation. A major component of this pre-brief is the discussion of selection parameters that are used to determine which projects and software activities will be a part of the survey. This pre-brief meeting is nominally held 6-8 weeks prior to the start of the survey. The actual survey event includes entrance presentations, document reviews by the OCE survey team, interviews of Center and project personnel by the survey team, development and review of initial findings, a review for surfacing general or systemic findings, and a summary presentation to members of management. The last activity is usually an exit briefing, which is the survey team's first formal opportunity to present their findings to the senior management of the surveyed organization.  A series of follow-on actions are defined and assigned.

As in many reviews, surveys, and appraisal activities, compliance is often measured against objective evidence. Expected content and types of this objective quality evidence are usually defined in the preparation for the software survey. This defined material provides the basis for confirmation of compliance with requirements and identification of strengths and weaknesses. The Carnegie Mellon University Software Engineering Institute provides the Capability Maturity Model Integration (CMMI) appraisal method

sweref
157
157
for process improvement.  The method provides an excellent discussion of the philosophy and types of objective evidence. The degree to which Centers follow the CMMI method can be assessed in a Standard CMMI Appraisal Method for Process Improvement (SCAMPI) activity. While the information presented in the SCAMPI discussion is centered on evaluating CMMI process implementation, the discussions and explanations in the CMMI text provide good background information for use in the OCE appraisals to people for who are relatively inexperienced in appraisals and surveys.


Panel

Findings resulting from the survey are generally classified as strengths, weaknesses, observations, opportunities, and non-compliances. See the Requirements Compliance Survey Process document

sweref
374
374
for definitions of these terms. However, the survey team has a clear and overriding obligation to identify all items of non-compliance and items that adversely affect safety or quality. These items will be included in the final report. Significant issues are brought to the immediate attention of the surveyed organization's management via the survey manager


Additional guidance related to OCE Appraisal activities may be found in the following related requirements in this Handbook:


SWE-004

OCE Benchmarking

SWE-036

Software Process Determination

SWE-108

Center SW Improvement Plan



Div
idtabs-4

4. Small Projects

Typically, the OCE includes a small project in the survey activities at a Center. The OCE survey leader will work with the Center SW POC to develop the appropriate level of survey involvement for small projects.


Div
idtabs-5

5. Resources


refstable

toolstable


Div
idtabs-6

6. Lessons Learned

No Lessons Learned have currently been identified for this requirement.