1. Requirements

3.2.2 The project manager‘s software cost estimate(s) shall satisfy the following conditions: 

1. 1. Covers the entire software life cycle.
   2. Is based on selected project attributes (e.g., assessment of the size, functionality, complexity, criticality, reuse code, modified code, and risk of the software processes and products).
   3. Is based on the cost implications of the technology to be used and the required maturation of that technology.
   4. Incorporates risk and uncertainty.
   5. Includes the cost for software assurance support.
   6. Includes other direct costs.

1.1 Notes

In the event of a decision to outsource, it is a best practice that both the acquirer (NASA) and the provider (contractor/subcontractor) be responsible for developing software cost estimates. For any class of software that has significant risk exposure, consider performing at least two cost estimates.  

1.2 Applicability Across Classes

Key:    - Applicable | - Not Applicable

2. Rationale

Credible software cost estimates are a key activity performed as part of a project and software life-cycle management (NPR 7150.2, NASA Software Engineering Requirements, Chapter 3).  Software cost estimates are required to meet NASA policies and have specific characteristics that make them credible and provide programs and projects with confidence in the cost of work to be performed.

3. Guidance

Credible software cost estimates for NASA are required to meet specific conditions.  When preparing cost estimates, the software acquirer and the software provider work together to develop cost estimates that meet the conditions described below.

The software cost estimate(s) and associated estimation data should be documented in a software cost driver sheet (a sample is available in Software Processes Across NASA (SPAN), accessible to NASA users from the SPAN tab in this Handbook) that supports the development of a parametric cost estimate.

Life Cycle Coverage

A credible software project cost estimate covers the entire software life cycle from concept to retirement, including maintenance, support (SWE-015) and associated project attributes such as annual percent lines of code changed. A breakout of cost by life cycle phase and activity is good estimating practice and cost estimates should be updated to reflect new assumptions.

Project Attributes

Software cost estimates are based on project attributes, including the estimated size of the software, functionality, complexity, criticality, and risk.  Other attributes to consider include use of reused and/or modified code, database size, documentation requirements, and required reusability.  Many of these attributes apply to the software being created, the processes used to create and verify that software, and the products created throughout the life cycle such as requirements documentation, architecture, detailed design, test plans and procedures, etc. (SWE-015). 

Technology Implications

Given the need to use the latest technology to achieve its missions, NASA cost estimates take into account the cost implications of the technology to be used and the required maturation of that technology.  When developing a software cost estimate, maintain all assumptions and data associated with the technology to be used on the software project, and maintain good data and records associated with any cost saving identified from the use of software technology.  Make sure that the software estimate maintains data showing cost required for maturation of any planned use of new technology.  New technology can be defined as any new software development environments or tools or software development associated with a new or unproven technology.

Risk and Uncertainty

Risk and uncertainty both relate to the same underlying concept – randomness. Risk is quantifiable; uncertainty arises from imperfect knowledge.

When developing a cost estimate, it is important to identify common software risks, assess their impact on the cost estimate, and make revisions to the estimate based on these impacts, e.g., incorporate allowances for uncertainty in sizing and other cost estimation model inputs.

Compile known major risks.

As you generate the software risk list, consider the following:

• What WBS elements are affected.
• When it would occur.
• Likelihood of occurrence.
• Impact (consequences).
• What it would cost to mitigate the impact.
• Stakeholder and user perspectives.

Risk and uncertainty can be estimated and analyzed in various ways:

• Risk matrices (e.g., 5x5).
• Expected risk (likelihood of impact).
• Monte Carlo methods (Monte Carlo methods use random numbers to obtain numerical solutions when analytical methods are too difficult to use)  to produce distributions.

The project team may add the results of the expected risk impact or identified mitigations to the bottom up estimate. The cost models have Monte Carlo algorithms which produce a cost distribution and the identified risks are used to determine which model inputs should be entered with a wider range.

Software Assurance Support

Support efforts such as software assurance and, when applicable, Independent Verification and Validation (IV&V) also have costs associated with them that are important to include in the software cost estimate.  Including software assurance in the cost estimate helps ensure that the project fully understands the software assurance activities that should be performed on the project and provides the appropriate level of funding for those activities.  It is important for software assurance and the project to discuss the risks associated with reducing or limiting the software assurance effort on a project which may be caused, in part, by limited funding.  

Other Costs

Other factors to consider when developing the cost estimate include:

• Procurement of equipment such as workstations, test beds, and simulators.
• Procurement or shared costs of tools such as compilers, licenses, and configuration management tools.
• Travel for reviews, customer interfaces, and vendor visits.
• Training.

NASA-specific software cost estimation process information is available in Software Processes Across NASA (SPAN), accessible to NASA users from the SPAN tab in this Handbook.

Additional guidance related to software cost estimates may be found in the following related requirements in this Handbook :

4. Small Projects

The conditions for cost estimates apply to small as well as large tasks but should be scaled to fit the size and scope of the task.  All estimates are made using some combination of the basic estimation methods of analogy or Cost Estimating Relationship (CER).  Whatever method is used, it is most important that the assumptions and formulas used be documented to enable more thorough reviews, and to make it easier to revise estimates at future dates when assumptions may need to be revised.  A documented estimate is the first line of defense against arbitrary budget cuts.  Small projects nominally use the analogy estimates methods if the development organization has data to support them.  Use of at least two estimates is always considered to be a best practices for small and large software estimation activities.  

5. Resources

5.1 Tools

Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN).

NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.

The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.

6. Lessons Learned

The NASA Lessons Learned database contains the following lesson learned related to cost estimation:

• Flight Software Engineering Lessons. Lesson Number 2218.  The engineering of flight software is a major consideration in establishing JPL project total cost and schedule because every mission requires a significant amount of new software to implement new spacecraft functionality. Constraints to the development and testing of software concurrent to engineering the rest of the flight system has led to flight software errors, including the loss of some missions. The findings of several JPL studies and flight mishap investigations suggest a number of recommendations for mitigating software engineering risk ⁵⁷².

Other Lessons Learned from experience (Hihn, Jairus M. JPL. 818.354.1248) and the JPL Cost Estimation Handbookavailable in SPAN include the following concepts:

• Common costs left out of cost estimates.
• Review preparation.
• Documentation.
• Fixing Anomalies and Engineering Change Requests (ECR)’s.
• Testing.
• Maintenance.
• Basic management and coordination activities.
• Technical leads do spend time doing management activities.
• Mission Support Software Components.
• Development and test environments.
• Travel.
• Training.