bannera

Book A.
Introduction

Book B.
7150 Requirements Guidance

Book C.
Topics

Tools,
References, & Terms

SPAN
(NASA Only)

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migration of unmigrated content due to installation of a new plugin


{alias:SWE-037} {tabsetup:1. The Requirement|2. Rationale|3. Guidance|4. Small Projects|5. Resources|6. Lessons Learned} {div3:id=tabs-1} h1. 1. Requirements
Wiki Markup
Tabsetup
1. The Requirement
1. The Requirement
12. Rationale
23. Guidance
34. Small Projects
45. Resources
56. Lessons Learned


Div
idtabs-1

1. Requirements

2.5.6

The

project

shall

define

the

milestones

at

which

the

software

supplier(s)

progress

will

be

reviewed

and

audited

as

a

part

of

the

acquisition

activities.

h2.

1.1

Notes

Known

contract

milestones

are

expected

to

be

included

in

the

resulting

contract.

h2.

1.2

Applicability

Across

Classes

Class

D

not

Safety

Critical,

and

Class

G

are

labeled

with

"P

(Center)."

This

means

that

an

approved

Center-defined

process

that

meets

a

non-empty

subset

of

the

full

requirement

can

be

used

to

achieve

this

requirement.

Class

F

is

labeled

as

"X

(not

OTS)"

which

means

that

the

project

is

required

to

meet

this

requirement

for

all

software

that

is

not

considered

off-the-shelf.

{applicable:asc=1|ansc=1|bsc=1|bnsc=1|csc=1|cnsc=1|dsc=1|dnsc=p|esc=1|f=*|g=p} {div3}{div3:id=tabs-2} h1. 2. Rationale For any software project, it is critical for management to review progress early and periodically to ensure the project remains on schedule, is progressing toward implementation of the requirements, and ultimately is addressing the customer's needs. It is also important for management to confirm periodically that the technical goals of the project are being achieved and that the technical direction of the project is appropriate (NPR 7123.1, NASA Systems Engineering Processes and Requirements).{sweref:041} Milestone reviews provide this type of management visibility into a project. For software development that is acquired (supplied by a contractor), having regular progress reviews is even more important since these reviews are the keys to ensuring the contractor understood and will provide the product that NASA requested and that meets NASA's requirements for safety, quality, reliability, etc. Milestone reviews can also serve to facilitate and ensure coordination between multiple development groups including development groups at multiple NASA Centers and contractors. {div3}{div3:id=tabs-3} h1. 3. Guidance For acquired software development, milestone reviews are incorporated into the contract because the contract is the binding document for contractor performance and deliverables. Regardless of whether the development is in-house or contracted, the development agreement needs to contain, among other key elements, surveillance activities including monitoring activities, reviews, audits, decision points, meetings, etc. Other items related to milestone reviews to include in the development agreement are: * Review periods for deliverables. * Time period for making corrections to resolve findings. * Formal reviews, such as those found in NPR 7123.1A {sweref:041}, NPR 7120.5D{sweref:082}, NPR 7120.7{sweref:264} (IT and Institutional Infrastructure) and NPR 7120.8{sweref:269}(Research and Technology). * Technical reviews. * Progress reviews. * Acceptance reviews. Consult Center guidance on the following topics as Center guidance may provide insights into reviews and review topics to be considered for inclusion in the {term:SOW} and contract: * Acquisition. * Planning. * Scheduling. * Process Monitoring and Control (PMC). See the [7.3 - Acquisition Guidance] and [7.9 - Entrance and Exit Criteria] topics in this Handbook for additional guidance on this topic. The references in [7.3 - Acquisition Guidance] may provide additional guidance on project milestone reviews and topics for consideration. Topic [7.9 - Entrance and Exit Criteria] describes inputs, material that will be reviewed, and outputs for each life-cycle milestone review which may be useful as input to the development of checklists for these reviews. Consult the NPR 7120 family of requirements documents for definitions of milestones and approaches for different project types.   Keep in mind that reviews not included in the contract, may be difficult to require of the contractor, so it is important to ensure the SOW and other contract elements are reviewed by the proper project management and/or technical authority for completeness. {div3}{div3:id=tabs-4} h1. 4. Small Projects Projects review plans and milestones are covered in NPR 7120.5, NPR 7120.7{sweref:264}, NPR 7120.8 and NPR 7123.1A{sweref:041}. Projects are to ensure that software components are a part of specific project milestone reviews. Small projects need to determine a review process that meets the project requirements and contains adequate content to provide the greatest insights into progress toward the project's technical goals and the technical direction of the project. For projects designated "small" by Center criteria, it may be possible to reduce the number of reviews to meet time and cost constraints. Keep in mind, however, that milestone reviews are not to be eliminated, as these reviews are critical checkpoints in the life cycle of the project. Projects covered by NPR 7120.7{sweref:264} and NPR 7123.1A{sweref:041}are required to follow specific project milestone reviews with software components. Small projects need to determine the set of reviews that provide the greatest insights into progress toward the project's technical goals and the technical direction of the project. {div3}{div3:id=tabs-5} h1. 5. Resources {refstable} \\ {toolstable}\\ {div3}{div3:id=tabs-6} h1. 6. Lessons Learned A documented lesson from the NASA Lessons Learned database notes the following: *Acquisition and Oversight of Contracted Software Development (1999). Lesson Number 0921:* Tailorable acquisition management and oversight processes for NASA contracted software development are essential to ensure that customers receive a quality product. A documented lesson from the NASA Lessons Learned database includes as a cause of the loss of a mission "the lack of a controlled and effective process for acquisition of contractor-developed, mission critical software." In this particular case, the quality of the contractor's product was not monitored as it would have been if the proper milestones for reviewing and auditing contractor progress were in place{sweref:528}. {div3} {tabclose}


applicable
ansc1
asc1
bnsc1
csc1
bsc1
esc1
cnsc1
dnscp
dsc1
f*
gp



Div
idtabs-2

2. Rationale

For any software project, it is critical for management to review progress early and periodically to ensure the project remains on schedule, is progressing toward implementation of the requirements, and ultimately is addressing the customer's needs. It is also important for management to confirm periodically that the technical goals of the project are being achieved and that the technical direction of the project is appropriate (NPR 7123.1, NASA Systems Engineering Processes and Requirements).

sweref
041
041
 Milestone reviews provide this type of management visibility into a project.

For software development that is acquired (supplied by a contractor), having regular progress reviews is even more important since these reviews are the keys to ensuring the contractor understood and will provide the product that NASA requested and that meets NASA's requirements for safety, quality, reliability, etc.

Milestone reviews can also serve to facilitate and ensure coordination between multiple development groups including development groups at multiple NASA Centers and contractors.


Div
idtabs-3

3. Guidance

For acquired software development, milestone reviews are incorporated into the contract because the contract is the binding document for contractor performance and deliverables. Regardless of whether the development is in-house or contracted, the development agreement needs to contain, among other key elements, surveillance activities including monitoring activities, reviews, audits, decision points, meetings, etc.

Other items related to milestone reviews to include in the development agreement are:

  • Review periods for deliverables.
  • Time period for making corrections to resolve findings.
  • Formal reviews, such as those found in NPR 7123.1A
    sweref
    041
    041
    , NPR 7120.5
    sweref
    082
    082
    , NPR 7120.7
    sweref
    264
    264
     (IT and Institutional Infrastructure) and NPR 7120.8
    sweref
    269
    269
    (Research and Technology).
  • Technical reviews.
  • Progress reviews.
  • Acceptance reviews.

Consult Center guidance on the following topics as Center guidance may provide insights into reviews and review topics to be considered for inclusion in the SOW (Statement of Work) and contract:

  • Acquisition.
  • Planning.
  • Scheduling.
  • Process Monitoring and Control (PMC).

See the 7.3 - Acquisition Guidance and 7.9 - Entrance and Exit Criteria topics in this Handbook for additional guidance on this topic. The references in 7.3 - Acquisition Guidance may provide additional guidance on project milestone reviews and topics for consideration. Topic 7.9 - Entrance and Exit Criteria describes inputs, material that will be reviewed, and outputs for each life-cycle milestone review which may be useful as input to the development of checklists for these reviews. Consult the NPR 7120 family of requirements documents for definitions of milestones and approaches for different project types.
 
Keep in mind that reviews not included in the contract, may be difficult to require of the contractor, so it is important to ensure the
SOW and other contract elements are reviewed by the proper project management and/or technical authority for completeness.


Div
idtabs-4

4. Small Projects

Project review plans and milestones are covered in NPR 7120.5

sweref
082
082
, NPR 7120.7
sweref
264
264
, NPR 7120.8
sweref
269
269
, and NPR 7123.1A
sweref
041
041
. Projects are to ensure that software components are a part of specific project milestone reviews. Small projects need to determine a review process that meets the project requirements and contains adequate content to provide the greatest insights into progress toward the project's technical goals and the technical direction of the project.


Div
idtabs-5

5. Resources


refstable



toolstable


Div
idtabs-6

6. Lessons Learned

A documented lesson from the NASA Lessons Learned database notes the following:

Acquisition and Oversight of Contracted Software Development (1999). Lesson Number 0921: Tailorable acquisition management and oversight processes for NASA contracted software development are essential to ensure that customers receive a quality product. A documented lesson from the NASA Lessons Learned database includes as a cause of the loss of a mission "the lack of a controlled and effective process for acquisition of contractor-developed, mission critical software." In this particular case, the quality of the contractor's product was not monitored as it would have been if the proper milestones for reviewing and auditing contractor progress were in place

sweref
528
528
.