This Topic provides assistance to the software development team for completing the qualification of flight software according to Agency-level requirements contained in the NPR 7150.2. Software qualification is the collection of activities performed in the software development life cycle to affirm the quality and safety of the final implemented software product. Qualification of software for flight is the process of demonstrating whether the software is capable of fulfilling the specified requirements, meets the user's needs and has assured quality. Qualification requirements are a set of criteria and conditions that have to be met in order to qualify a software product as complying with its specifications and being ready for use in its target environment. Completion of all the software qualification activities affirms the quality and safety of the final implemented software product. Successful implementation of a Software Qualification Process ensures that the following items have been addressed: The qualification of software for flight includes those activities related to planning, configuration management, verification, validation, software testing, software integration, system testing, software analyses, software assurance, reviews, and acceptance. Many of the requirements in the NPR provide specific guidance for flight software qualification activities (see Table 1 in Section 4 below). The execution, completion, and approval of these activities are governed by the plans and procedures that are applicable to each of the activities separately. Even though the Guidance entries for these software development activities are presented as separate essays (SWEs, Topics) in the handbook, these activities all need to be planned and completed together to assure that the qualification of the flight software is accomplished. The NPR includes the minimum set of requirements to protect the Agency's investment in software engineering products. This Topic supplements the information in the NPR by emphasizing the key requirements affecting qualification and by providing some explanatory notes and brief suggestions for evaluating the application or intent of the requirements to the software development qualification activities. One of the final activities in the software development process is the qualification of flight software and hardware as an integrated system. The basis for all of the qualification efforts is the planning activity. The planning and process flow for the qualification of flight software can be depicted notionally by the pyramid structure of Figure 1. The basic activities are separate actions, but the production of quality products depends on the successful implementation of all of them together. The key feature of flight software qualification is the assessment that all necessary activities (V&V, testing, assurance, analyses, reviews, and configuration management) have been completed successfully. One of the many review activities, the System Acceptance Review, signals the nominal completion of these flight software qualification activities. When attempting to understand what qualification means, consider whether all the development, test, review, and acceptance activities that were planned for actually did occur, that the results are acceptable and currently available, and that all appropriate and required documentation is complete, available, reviewed, and properly stored. Figure1. Flight Software Qualification Pyramid The project team periodically evaluates the qualification activities to assess their correctness and degree of completion. These evaluations typically occur over a series of reviews and or related activities, and across several phases of the software development life cycle. Numerous references that discuss reviews and qualification activities, including NASA documents and external sources, are provided in the Resources section below. Two checklist tools are also included in the Resources section. One summary of a NASA approach for assessing the qualification of flight software that uses a prescribed series of reviews is described in Nelson. 398 The above quotes from Nelson indicate in the terms 'internal project team', 'independent team of engineers', 'Program Manager', and 'Final Reviewers' that a high degree of collaboration is necessary to ensure a successful qualification activity. Another example states that the System Acceptance Review (SAR) completes the qualification activities for the flight software. The SAR is conducted by the project team to assure the software product has completed all activities prior to preparing the product for shipping. Some consider this to be a pre-ship review. It usually occurs before the Readiness Reviews. 041 Numerous reviews that occur during the software development life cycle (see SWE-019) provide interim assessments of the readiness and completeness of the software components, sub-systems, and integrated systems. These reviews generate results that all feed into the necessary qualification activities. For many of them, 'Entrance and Success' (Exit) criteria are necessary to begin or complete the various reviews and life cycle phases. See Topoic 7.09 - Entrance and Exit Criteria for listings and a detailed discussion of Entrance and Success criteria for the various reviews. The satisfaction of these entrance and/or success criteria in the listings provide additional confidence that the software work products are built with quality, are complete, and are ready for flight operations. The NASA Systems Engineering Processes and Requirements, NPR 7123.1A, 041 and the NASA Systems Engineering Handbook,NASA SP-2007-6105, 273 provide more detailed guidance on reviews. These formal and informal reviews are separately and collectively informed by the plans, activities, and or results gained from the software development activities. As indicated in the Figure 1, five major process areas, when properly executed, together generate the information for the reviews. Major activities that need to be completed to ensure the successful qualification of flight hardware are discussed in the following paragraphs. As you assess the qualification of the flight software on your project, consider the following associated 'assistance thoughts' and assess if the software is ready. 1. Planning - covers the software development on a project from inception through retirement. The processes and activities are coordinated within the project. Software needs for the project for acquisition, supply, development, operations, maintenance and retirement are developed. During this activity, consider the following: 2. Configuration Management - covers the reviews and checks that all necessary items have been completed and incorporated into the product, that all identified errors and corrections have been made, that all documentation has been completed, and that all products have been evaluated, tested, and found acceptable. During this activity, consider the following: 3. Software Assurance - covers the assurance planning that independently ensures conformance of software life cycle processes and products to requirements, standards, and procedures. Software assurance ensures that all processes used to acquire, develop, assure, operate and maintain the software are appropriate, sufficient, planned, reviewed, and implemented according to plan, meet any required standards, regulations, and quality requirements. Software assurance assures that the software and its related products meet their specified requirements, conform to standards and regulations, are consistent, complete, correct, safe, secure and reliable as warranted for the system and operating environment, and satisfy customer needs. Software assurance utilizes relevant project-based measurement data to monitor each product and process for possible improvements. During this activity, consider the following: 4. Verification & Validation (V&V) - covers the development activities that ensure that software being developed or maintained satisfies functional and other requirements and that each phase of the development process yields the right products. V&V processes determine whether the development products of a given activity conform to the requirements of that activity and whether the product satisfies its intended use and user needs. This determination may include analysis, evaluation, review, inspection, assessment, and testing of products and processes. The dynamics of an integrated system software product demand that the V&V effort examine the correctness of the software product for each possible variation in conditions. The ability to model these complex real world conditions will be limited, however, and thus the V&V effort may have to limit itself to assessing whether the limits of the modeling are realistic and reasonable for the desired solution. Besides the project's V&V activities, Independent V&V activities may use different operators, different hardware, simulation systems, and test procedures. They may use some developer tests or some customer generated tests. To qualify any software, a consistent logical progression of testing, in accordance with good software engineering practice, starting with unit testing and progressing to integration testing, and then to system testing, must be completed. They need to include path and thread testing, off-nominal cases, boundary conditions, stress cases, and robustness testing. Testing produces documented evidence useful in reaching understanding and conclusions about the true characteristics of the items tested. During this activity, consider the following: 5. Acceptance Review – covers the numerous acceptance activities. One of these, the formal System Acceptance Review, verifies the completeness of the specific end products in relation to their expected maturity level and assesses compliance to stakeholder expectations. The SAR examines the system, its end products and documentation, and test data and analysis that support verification. It also ensures that the system has sufficient technical maturity to authorize its shipment to the designated operational facility or launch site. During this activity, consider the following: Safety considerations must be monitored in all qualification activities. Consider safety review packages as a special case of material needed for the qualification of flight software. See SWE-022 and SWE-023. Remember that regulatory authorities will be looking for evidence that all potential hazards have been identified and that appropriate steps have been taken to deal with them. Additional guidance for qualification of flight software may be found in the following applicable and inter-related requirements in this handbook: Software qualification planning V&V activities Software testing Change tracking and resolution Software classification & criticality Acceptance Documentation Tools to aid in compliance with this Topic, if any, may be found in the Tools Library in the NASA Engineering Network (NEN). NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN. The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool. The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.
View this section on the website
See edit history of this section
Post feedback on this section
1. Purpose
1.1 Introduction
2. Guidance
3. Safety
4. Additional Guidance
5. Resources
5.1 Tools
7.12 - Qualification of Flight Software
Web Resources
Unknown macro: {page-info}
