See edit history of this section
Post feedback on this section
1. Purpose
This topic discusses the relationship between the requirements and associated processes for NPR 7150.2, NASA Software Engineering Requirements 083, and the content of NASA-STD-7009, Standard for Models and Simulation. Because NASA-STD-7009 is generally applicable to all types of models and simulations (which are most commonly embodied in software), it is important to understand its relevance to NPR 7150.2 and the precedence to be maintained between the two documents. Software developers can use this topic to understand the relevance and applicability of NASA-STD-7009 when developing software under NPR 7150.2.
1.1 Introduction
1.1.1 Relevant Guidance in NPR 7150.2
As discussed elsewhere, the NPR 7150.2 requirements applicability to the software being developed is determined by the contents of its Appendix C, Requirements Mapping Matrix. Requirement SWE-139 implements the contents of Appendix C. NPR 7150.2 applies to the development of software models and simulations. The NASA-STD-7009, in turn, describes how models and simulations are to be verified within the bounds of the governing SWE requirements of the NPR 7150.2.
NPR 7150.2 references the content of NASA-STD-7009 via requirement SWE-070 in the presentation of the requirement in its section 4.5.6 and the accompanying Note:
"4.5.6 The project manager shall use validated and accredited software models, simulations, and analysis tools required to perform qualification of flight software or flight equipment." (See SWE-070)
Note: Information regarding specific V&V techniques and the analysis of models and simulations can be found in NASA-STD-7009, Standard for Models and Simulations, NASA-HDBK-7009, Handbook for Models and Simulations, or discipline-specific recommended practice guides.
1.1.2 Applicability and Scope of NASA-STD-7009
The applicability and scope are stated in section 1.2 of NASA-STD-7009, where the acronym "M&S" denotes "models and simulations":
This NASA Technical Standard establishes uniform practices in modeling and simulation to ensure essential requirements are applied to their design, development, and use, while ensuring acceptance criteria are defined by the program/project and approved by the responsible Technical Authority.
This NASA Technical Standard provides an approved set of requirements, recommendations, and criteria with which models and simulations (M&S) may be developed, accepted, and used in support of NASA activities. As the M&S disciplines employed and application areas involved are broad, the common aspects of M&S across all NASA activities are addressed. The discipline-specific details of a given M&S should be obtained from relevant recommended practices.
1.1.3 Implications for Other Models and Simulations
For all other models and simulations that are deemed by the M&S Risk Assessment to be in the scope of NASA-STD-7009, there is the need to ensure that the requirements of both documents are satisfied. From the perspective of NASA-STD-7009, some of the requirements in NPR-7150.2 are not related to M&S, some are supplemental to requirements in NASA-STD-7009, and others are subsets of requirements in NASA-STD-7009.
1.1.4 Rationale for STD-7009
The NASA Standard for Models and Simulations (NASA-STD-7009) had its genesis in the Space Shuttle Columbia Accident Investigation (2003). Generally, its purpose is to improve the "development, documentation, and operation of models and simulations" (Diaz Report) and per a September 2006 memo from the Office of the Chief Engineer "which include a standard method to assess the credibility of the models and simulations". After an approximately three-year development period, the NASA Standard for Models and Simulations, NASA-STD-7009 was approved by NASA's Engineering Review Board on July 11, 2008, for voluntary use.
NASA-STD-7009 holds a unique place in the world of modeling and simulation in that it is, by direction, generally applicable to all types of models and simulations (M&S) and in all phases of development, though it is primarily focused on the results of an M&S-based analysis. All standards and recommended practices for M&S to date have either been focused on a single type of M&S (e.g., structures, fluids, electrical controls, etc.) or a particular phase of M&S development (e.g., verification, validation, etc.). NASA management is confronted with numerous types of analyses that may be involved in making critical decisions. Depending on the situation at hand, a common framework for understanding the results and assessing the credibility of that analysis may seem intuitive. However, this is complicated by the vast differences in engineering systems, and, thus, the adoption of a standard like this has been slow.
After formal approval in July 2008, and the update in 2016, the NASA-STD-7009 was largely left to the individual program, project, or M&S practitioner to adopt as they wished. While already existing programs and projects were not required to adopt it, new programs and projects were to adopt it, depending on their needs, desires, and criticality of the M&S-based analysis at hand.
1.1.5 Guidance for NASA-STD-7009
Guidance for use and application of NASA-STD-7009 can be found in the NASA-HDBK-7009 Handbook 248. This is a comprehensive instruction set on the use and application of NASA-STD-7009 as it relates to the Verification and Validation of Models and Simulations.
2. Resources
2.1 References
- (SWEREF-005) AIAA-2008-2156 Blattnig, S.R.; Green, L.L.; Luckring, J.M.; Morrison, J.H.; Tripathi, R.K.; Zang, T.A. (2008). NASA users can access AIAA documents via the NASA Technical Standards System located at https://standards.nasa.gov/. Once logged in, search to get to authorized copies of AIAA documents.
- (SWEREF-009) NPR 8000.4B, NASA Office of Safety and Mission Assurance, 2017. Effective Date: December 06, 2017 Expiration Date: December 06, 2022
- (SWEREF-082) NPR 7120.5F, Office of the Chief Engineer, Effective Date: August 03, 2021, Expiration Date: August 03, 2026,
- (SWEREF-083) NPR 7150.2D, Effective Date: March 08, 2022, Expiration Date: March 08, 2027 https://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=7150&s=2D Contains link to full text copy in PDF format. Search for "SWEREF-083" for links to old NPR7150.2 copies.
- (SWEREF-124) ASME (2006). New York, NY Available to members
- (SWEREF-126) Balci, O. (2004). Proceedings of the 2004 Winter Simulation Conference. R.G. Ingalls; M.D. Rossetti; J.S. Smith; B.A. Peters, eds. Dec. 5-8. Piscataway, NJ: IEEE. pp. 122-129.
- (SWEREF-128) Banks, J., ed. (1998). New York: John Wiley & Sons, Sep 14, 1998,
- (SWEREF-144) Diaz, Al,NASA Goddard Space Flight Center (Jan, 2004). CAIB Columbia Accident Investigation Board Report. (August 2003). Vol. 1. PB2005-100968
- (SWEREF-155) Clemen, R.T. (1996). Second Edition. Pacific Grove, CA: Brooks/Cole. download available from link.
- (SWEREF-164) Cooke, R.M. (1991). New York: Oxford University Press.
- (SWEREF-187) Fidelity ISG Glossary Simulation Interoperability Standards Organization (SISO). (Dec. 1998). Vol. 3.0. Site still exists at https://www.sisostds.org/. Unable to locate the Fidelity ISG Glossary.
- (SWEREF-198) AIAA G-077. Reston, VA: AIAA. 1998. This document is avaible from AIAA; NASA has access to the AIAA website via the NASA START (AGCY NTSS) system (https://standards.nasa.gov ).
- (SWEREF-200) Hale, J.P.; Hartway, B.L.; Thomas, D.A. (2007). The 5th Joint Army-Navy-NASA-Air Force (JANNAF) Modeling and Simulation Subcommittee Meeting, May, CDJSC 49. Columbia, MD: Johns Hopkins University.
- (SWEREF-202) Harmon, S.Y.; Youngblood, S.M. (2005). Vol. 2, No. 4, pp. 179-190. Conference: Spring 2003 SISO Simulation Interoperability Workshop, At Kissimmee, FL
- (SWEREF-222) IEEE STD 610.12-1990, 1990. NASA users can access IEEE standards via the NASA Technical Standards System located at https://standards.nasa.gov/. Once logged in, search to get to authorized copies of IEEE standards.
- (SWEREF-240) Lin, J.; West, J.S.; Williams, R.W.; Tucker, P.K. (2005). AIAA-2005-4524 .
- (SWEREF-248) NASA-HDBK-7009 Revision: A, Document Date: 2019-05-08, Next 5-Year Review Date: 2024-05-08
- (SWEREF-251) Mehta, U.B. (2007). The 5th Joint Army-Navy-NASA-Air Force (JANNAF) Modeling and Simulation Subcommittee Meeting, CDJSC 49, May, CPIAC. Columbia, MD: Johns Hopkins University.
- (SWEREF-271) NASA STD 8719.13 (Rev C ) , Document Date: 2013-05-07
- (SWEREF-272) NASA-STD-7009A w/ CHANGE 1: ADMINISTRATIVE/ EDITORIAL CHANGES 2016-12-07
- (SWEREF-286) NPR 8715.3D, Effective Date: December 16, 2021, Expiration Date: December 21, 2026
- (SWEREF-288) Oberkampf, W.L.; Pilch, M.; Trucano, T.G. (October 2007), SAND2007-5948. Sandia National Laboratories: Albuquerque, New Mexico 87185 and Livermore, California 94550.
- (SWEREF-311) Defense Modeling and Simulation Office. http://vva.dmso.mil/.
- (SWEREF-312) Saltelli, A.; Chan, K.; Scott, E.M., eds. (2000). Chichester, England: John Wiley & Sons. Available via http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471998923.html.
- (SWEREF-414) Babula, M. et al. (2008). Published Online:15 Jun 2012 https://doi.org/10.2514/6.2009-1011
- (SWEREF-460) NPR 7150 traced to NASA-STD-8739 8 and NASA-STD-8719 13B_20100924 This NASA-specific information and resource is available in Software Processes Across NASA (SPAN), accessible to NASA-users from the SPAN tab in this Handbook.
- (SWEREF-582) Public Lessons Learned Entry: 377.
2.2 Tools
NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN.
The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.
3. Lessons Learned
3.1 NASA Lessons Learned
A documented lesson from the NASA Lessons Learned database notes the following:
- Performance Decrease due to Propulsion Thruster Plume Impingement on the Voyager Spacecraft, Lesson Number: 0377 582: "A 21% shortfall in Voyager's velocity change was suspected to be due to exhaust plume impingement. Due to the complexity of spacecraft/thruster configurations, additional care must be taken in the development and utilization of spacecraft and plume models. Analysis should be conducted on early and final designs.".
3.2 Other Lessons Learned
The requirements for a NASA Standard have matured from the Columbia Accident Investigation Board (CAIB) Report. The CAIB report found problems pertaining to "ineffective and inconsistent application of M&S tools, along with cases of misuse." It called on NASA to "develop, validate, and maintain physics-based computer models to evaluate Thermal Protection System damage from debris impacts. These tools should provide realistic and timely estimates of any impact damage from possible debris from any source that may ultimately impact the Orbiter." NASA was to establish impact damage thresholds that trigger responsive corrective action, such as on-orbit inspection and repair, when indicated.
Lessons Learned and their applicability to the need to perform verification and validation are well documented in the above identified resources. These should be reviewed and retained by any program or project utilizing Modeling and Simulation products throughout the Software development life cycle.