The project risk is the high risk of undiscovered software defects and low code quality. Any code that has not been through a code peer review has a high risk of undiscovered software defects and low quality. Software peer reviews or inspections are performed to ensure product and process quality, add value, and reduce the project risk through expert knowledge infusion, confirmation of approach, identification of defects, and specific suggestions for product improvements. Code peer reviews are one of the best ways to identify errors and problems in the code when the reviews follow practices such as including people with different roles in the project participating and spending adequate time reviewing the code in advance of the meetings. This technique should be used particularly on the mission-critical code to reduce errors or problems with the code. This technique supplements the use of tools and testing for errors and often allows the errors to be found earlier when they are cheaper to fix. Generally, fewer errors and problems are found in critical code when a higher percentage has been peer-reviewed. A substantial body of data and experience justifies the use of inspections on requirements. Finding and fixing requirements problems during requirements analysis is cheaper than doing so later in the life cycle and is substantially cheaper than finding and fixing the same defects after delivering the software.

The project risk is the high risk of undiscovered software defects, low code quality, missed schedule milestones, and increased software operational costs. Any code that multiple static analysis tools (more than 4 tools) have not analyzed has a high risk of undiscovered software defects and low quality. Studies show that the cost of catching errors dramatically increases from one phase of development to the next by roughly a factor of 10. The static analysis requirement for NASA software projects increases the quality and safety of code developed for NASA Missions. Using static analysis helps to ensure that code meets the coding standards/criteria established by the project team and contributes to the identification of common coding errors before system integration and testing (SWE-066 - Perform Testing). Studies show that the cost of catching errors dramatically increases from one phase of development to the next by roughly a factor of 10. Eliminating errors during implementation results in cost and time savings during integration and testing, which is a particularly important cost-saving factor for projects using high-fidelity testbeds.  Using static analysis helps to ensure that code meets the coding standards/criteria established by the project team and finds common coding errors that need to be eliminated before system integration and testing. Eliminating errors during implementation results in cost and time savings during integration and testing, which is a particularly important cost-saving factor for projects using high-fidelity testbeds.

The project risk is extended project testing time required, inconsistent system and software test results, and higher software cost. Testing software without using a software configuration management system will lead to extended project testing and inconsistent system and software test results. Configuration Management is the process of maintaining systems, such as computer hardware and software, in a desired state. Configuration Management (CM) is also a method of ensuring that systems perform in a manner consistent with expectations over time. Software Configuration Management planning encompasses the practices and procedures for administering source code, producing software development builds, controlling change, and managing software configurations for all of the software products, tools, data, and components produced by the project. As software teams design, develop and deploy software, it is common for multiple versions of the same software to be used on different sites and for the software developers to be working simultaneously on updates. Bugs or defects in the software are often only present in certain versions (because of the fixing of some problems and the introduction of others as the program develops). Therefore, to locate and fix bugs, it is important to be able to retrieve and run different versions of the software to determine in which version(s) the problem occurs. It may also be necessary to develop two versions of the software concurrently (for instance, where one version has bugs fixed, but no new features, while the other version is where new features are developed.  Change requests address not only new or changed requirements but also failures and defects in software products. Change requests are analyzed to determine the impact that the change will have on the software product, related software products, the budget, and the schedule. Tracking and evaluating changes are useful for a variety of reasons, not the least of which is to maintain documented descriptions of problems, issues, faults, etc., their impact on the software and system, and their related resolutions. Evaluating changes allows key stakeholders to determine the cost-benefit of implementing changes and to make decisions based on that information.

The project risk is the high risk of undiscovered software defects due to not following the processes, low code quality, missed schedule milestones, and increased software operational costs. Software products are engineered under the practice of using processes to improve the quality of a software development effort and a methodical approach to software development results in fewer defects.  The increased complexities and huge size of the software development projects make software development hard to control without formal practices and standard measurements. The risk is that the NASA project is supported by a software development organization that has the necessary skills and processes in place to produce reliable products within cost and schedule estimates

The project risk is that the software has security vulnerabilities that could compromise software security and lead to project failure. Secure coding practices can help prevent cyber attacks by eliminating vulnerabilities and weaknesses in software code. Unsafe coding practices result in costly vulnerabilities in application software that lead to the theft of sensitive data.   Some secure practices include but are not limited to the strict language adherence and use of automated tools to identify problems either at compile time or run time; language-specific guidance, domain-specific guidance, local standards for code organization and commenting and standard file header format are often specified as well.

The project risk is the high risk of undiscovered software defects, low code quality, missed schedule milestones, and increased software operational costs. Without using a good coding standard. code will not have a uniform appearance when written by different engineers. Coding standards improve readability and maintainability of the code and it reduces complexity. It helps in code reuse and helps to detect errors easily. It promotes sound programming practices and increases the efficiency of the programmers. A coding standard gives a uniform appearance to the codes written by different engineers. It improves the readability and maintainability of the code and it reduces complexity also. It helps in code reuse and helps to detect errors easily. It promotes sound programming practices and increases the efficiency of the programmers.

The project risk is the high risk of major software defects, compiler warnings are messages generated by the compiler to alert the developer about potential problems or issues with their code. Warnings report other unusual conditions in your code that may indicate a problem, although compilation can (and does) proceed. Compiler warnings are messages produced by a compiler regarding program code fragments to be considered by the developer, as they may contain errors.

The project risk is the high risk of major software failure and loss of the project. A common-mode primary Flight Software fault due to a coding/logic error, a processor resource overrun, a database error, or a computer virus may result in Loss of Vehicle (LOV). The risk is especially high during dynamic flight phases when there is a short time-to-effect, or when faster-than-human response times are required. Safe Mode coverage during on-orbit quiescent phases of flight, as well as backup flight software (BFS) coverage during, entry, descent, and landing (EDL) operations, provide additional layers of protection against LOV in the event of a common-mode primary Flight Software fault. A credible risk of common cause failures exists for fail-silent and erroneous-output cases. In human spaceflight systems, software commonly causes errors and introduces a risk such that a single failure in flight software can result in a catastrophic loss-of-vehicle-control hazard. 

The project risk is incomplete testing of safety-critical software functionality needed to protect against loss of crew or vehicle. All Safety-critical software decisions must be tested to protect against loss of crew or vehicle.  MCDC testing represents the minimal set of tests necessary to achieve test coverage over decisions that change the behavior/outcome/output of a computer program.   Anything less than MCDC exposes a risk of a safety-critical decision based on a set of conditions not being tested. Aerospace and space guidance prioritizes safety above all else in the software development life cycle. MC/DC represents a compromise that finds a balance between rigor and effort, positioning itself between decision coverage (DC) and multiple condition coverage (MCC). MC/DC requires a much smaller number of test cases than multiple condition coverage (MCC) while retaining a high error-detection probability. Risk is incomplete testing, all Safety-critical software decisions must be tested to protect against loss of crew or vehicle.

The project risk is that the software has security vulnerabilities that could compromise software security and lead to project failure. Cybersecurity vulnerabilities and weaknesses identified in a system indicate that there are insecure areas in the code where critical information could be altered or where control functions could be taken over or altered. Vulnerabilities and weaknesses that are found need to be mitigated to avoid security problems. The more vulnerabilities and weaknesses that are identified, the higher the risk that security issues may cause problems with the system's operations.

The project risk is that the test code may differ from the flight code. A test as you fly risk. The risk associated with using a different compiler for the test code is that the test code may not be the same as the flight code.  Using a different compiler may cause compiler problems in the flight code.  The test code may be different than the flight code.

The project risk is the high risk of undiscovered software defects, low code quality, missed schedule milestones, and increased software operational costs. Coding standards help ensure the safety, security, reliability, quality, maintainability, readability, and testability of the NASA code products. NASA programs and projects have multi-year life cycle times. Often the software personnel who develop the original software work products move on to other projects. These developers are then backfilled on the team by other developers for the remainder of the development, operations, maintenance, and disposal phases of the life cycle. This personnel turnover process may occur several times during the project's life cycle. The use of uniform software coding methods, standards, and/or criteria ensures uniform coding practices, reduces errors through safe language subsets, and improves code readability. Verification that these practices have been adhered to reduces the risk of software malfunction for the project during its operations and maintenance phases. There are five key benefits of using coding standards:

1. Understanding, maintainability, and readability of the code;
2. Consistent code quality — no matter who writes the code;
3. Software security from the start;
4. Reduced development costs;
5. Testability of the code.

Coding standards help prevent or reduce unsafe coding practices such as defect-prone coding styles, security issues from specific coding sequences, and numerous coding errors, mistakes, and misunderstandings. 

The risk is incomplete software testing and reduced reliability associated with safety-critical software code components. The intent is to minimize risk, minimize testing, and increase reliability associated with safety-critical software code components, thus reducing the chance of software failure during a hazardous event. A section of source code's cyclomatic complexity is the number of linearly independent paths within it—where "linearly independent" means that each path has at least one edge that is not in one of the other paths.

Requirements volatility is one of the leading causes of the software development effort not completing on schedule and budget.  Software requirements volatility is one key factor in assessing the status of a software project.  The later in the project requirements changes occur, the more impact those changes can have on project completion on time and within budget. 

The software should have demonstrable correctness properties, supported by evidence from the use of appropriate static analysis techniques.

Unit test procedures are to be repeatable so that future runs can confirm that any identified flaws have been corrected and for regression purposes to ensure that any new changes do not introduce new flaws in the software. Unit testing can be described as the confirmation that the unit performs the capability assigned to it, correctly interfaces with other units and data, and represents a faithful implementation of the unit design.

Code peer reviews are one of the best ways to identify errors and problems in the code when the reviews follow practices such as including people with different roles in the project participate, and spend adequate time reviewing the code in advance of the meetings. This technique should be used particularly on the mission critical code to reduce the error or problems with the code. This technique supplements the use of tools and testing for errors, and often allows the errors to be found earlier when they are cheaper to fix. Generally fewer errors and problems are found in critical code critical code when a higher percentage  has been peer reviewed. A substantial body of data and experience justifies the use of inspections on requirements. Finding and fixing requirements problems during requirements analysis is cheaper than doing so later in the life cycle and is substantially cheaper than finding and fixing the same defects after delivering the software.

Risk is incorrect software control of hardware components. Peer reviews are focused, in-depth technical reviews that support the evolving design and development of a product, including critical documentation or data packages. The participants in a peer review are the technical experts and key stakeholders for the scope of the review. The software peer reviews should be an independent evaluation by internal or external subject matter experts who do not have a vested interest in the work product under review. Projects can plan peer reviews, and focused reviews conducted on selected work products by the producer’s peers to identify defects and issues before that work product moves into a milestone review or approval cycle

Risk is errors in the flight software development tools create errors and defects in the flight software product. Performing verification and validation (V&V) to accredit software models, simulations, and analysis tools is important to ensure the credibility of the results produced by those tools. Critical decisions may be made, at least in part, based on the results produced by models, simulations, and analysis tools. Reducing the risk associated with these decisions is one reason to use accredited tools that have been properly verified and validated. Performing V&V to approve Software models, simulations, analysis tools, and software tools is important to ensure the credibility of those tools’ results and how the tool directly affects the Software being developed. Critical decisions may be made, at least in part, based on the results produced by these tools. Reducing the risk associated with these decisions is one reason to assure that the approved tools have been properly verified and validated. Information regarding specific 

Risk is incomplete interface definitions and misunderstanding of the data use in the software algorithms.

The design addresses the software architectural design and software detailed design.  The objective of doing design analysis is to ensure that the design:  is a correct, accurate, and complete transformation of the software requirements that will meet the operational needs under nominal and off-nominal conditions, is safe, is secure with known weaknesses and vulnerabilities mitigated, introduces no unintended features, and does not result in unacceptable operational risk.

Software architecture deals with the fundamental organization of a system, as embodied in its components and their relationships to each other and the environment. Software architecture reviews are conducted to inspect quality attributes, principles of design, verifiability, and operability of a software system. The software architecture is reviewed to evaluate its effectiveness, efficiency, and robustness relative to the software requirements.  The software architecture review helps manage and/or reduce flight software complexity through improved software architecture, improved mission software reliability and cost savings. The software architecture underpins a system's software design and code; it represents the earliest design decisions, ones that are difficult and costly to change later. The transformation of the derived and allocated requirements into the software architecture results in the basis for all software development work. Software architecture: Formalizes precise subsystem decompositions; Defines and formalizes the dependencies among software work products within the integrated system; Serves as the basis for evaluating the impacts of proposed changes; Maintains rules for use by subsequent software engineers that assure a consistent software system as the work products evolve; and provides a stable structure for use by future groups through the documenting of the architecture. its views and patterns, and its rules.

Missing capabilities to monitor key software observables (e.g. number of failed login attempts, performance changes, internal communication changes) to detect adversarial actions that threaten mission success.

Any uploaded or uplinked data, rules, and code can affect the behavior of the software and/or system. Special acceptance tests should be developed to validate and verify the uplinked or uploaded information for nominal and off-nominal scenarios.

The risk is that the software will crash in conditions of corrupted commands, data, or loads, and memory faults allocated to the software. 

Risk is incomplete software components and untested software capabilities. Requirements are the basis for a software product. They identify the needs to be addressed, the behavior of the system, the desired quality of the software, and the constraints under which the problem is to be solved. They specify the product the provider is to deliver to a customer.  Requirements serve as the basis for verification activities allowing the developing organization and the customer to judge the completeness of the product.

Risk is incomplete software components and untested software capabilities. All changes need to be addressed in the are the software products.  They specify the product the provider is to deliver to a customer.  

Risk is misuse or misfunctioning of the software capabilities and functionality. Very few software projects result in a product that users understand and can use efficiently, completely, and successfully without some guidance. Software documentation "explains the capabilities of the software or provides operating instructions for using the software to obtain the desired results." Along with software and other documentation, the final delivered package for software projects, including contracted projects, includes operational instructions for the delivered software. These instructions are typically provided by a Software User Manual, which contains both user instructions and a description of the functions and features provided by the software.

Software Configuration Management planning encompasses the practices and procedures for administering source code, producing software development builds, controlling change, and managing software configurations for all of the software products, tools, data, and components produced by the project. 

Risk of not performing IV&V is potential software defects and a lack of rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.) and off-nominal conditions (response to faults, responses to hazardous conditions, etc.).IV&V is a technical discipline of software assurance that employs rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.), and off-nominal conditions (response to faults, responses to hazardous conditions, etc.). The goal of the IV&V effort is to contribute to the assurance conclusions to the project and stakeholders based on evidence found in software development artifacts and risks associated with the intended behaviors of the software.

Risk is that the software development practices and processes are not being followed and that software defects may exist as result of not following or upstanding the software processes

Risk is unclear guidance on milestone reviews and final software product capability, functionality, and testing.  Acceptance criteria is an important component. It clearly defines the scope, desired outcomes of, and testing criteria for pieces of functionality that the delivery team is working on. The process of creating and agreeing on acceptance criteria itself is also an invaluable communication opportunity between developers and product.

Risk is missing hazards associated with software and mechanizations' of software safety criticality. It is important to determine the safety criticality of each software component to identify the most critical software system components and to ensure that the software safety-critical requirements and processes are followed. Software safety analysis supplements the system hazard analysis by assessing the software performing critical functions serving as a hazard cause or control. A typical software safety analysis process identifies the must work and must not work functions in the hazard reports. The system hazard analysis and software safety analysis process should assess each function for compliance with the levied functional software requirements. The system hazard analysis and software safety analysis also assure the redundancy management performed by the software supports fault tolerance requirements. 

Risk of a current or former employee, contractor, vendor or partner with legitimate user credentials misuses their access to the detriment of the organization's software, networks, systems or data.

Risk is that the software development does not have the necessary people in place, skills and processes in place to produce reliable products within cost and schedule estimates. 

Risk is unusable software or high complexity in software operations

Risk is incomplete hazard verification

Risk of not performing IV&V is potential software defects and a lack of rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.) and off-nominal conditions (response to faults, responses to hazardous conditions, etc.).IV&V is a technical discipline of software assurance that employs rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.), and off-nominal conditions (response to faults, responses to hazardous conditions, etc.). The goal of the IV&V effort is to contribute to the assurance conclusions to the project and stakeholders based on evidence found in software development artifacts and risks associated with the intended behaviors of the software.

Risk is that NASA missions may not be resilient to purposeful threats. Current trends such as technology proliferation, accessibility to space, globalization of space programs and industries, commercialization of space systems and services, and foreign knowledge about U.S. space systems increases the likelihood that the U.S. will experience a "Space Pearl Harbor." For example, in July 2000, the Xinhua news agency reported that China's military is developing methods and strategies for defeating the U.S. military in a high-tech and space-based future war. It noted, "For countries that could never win a war by using the method of tanks and planes, attacking the U.S. space system may be an irresistible and most tempting choice..."(2) These reports illustrate an unpleasant but little noticed view of the future. The ability to restrict or deny freedom of access to and operations in space is no longer limited to global military powers. The reality is that there are many extant capabilities to deny, disrupt or physically destroy space systems and the ground facilities that command and control them. Knowledge of U.S. space systems functions, locations and physical characteristics, as well as the means to conduct counterspace operations, is increasingly available on the international market. Nations or groups hostile to the U.S. possess or can acquire the means to disrupt or destroy U.S. space systems by attacking the satellites in space, their communications nodes on the ground and in space, or ground nodes that command the satellites.

Command link incidents with civil space missions have demonstrated potential impacts to safe operations. Additionally, NASA end of mission (EOM) experiments found that spacecraft without encryption or authentication are particularly susceptible to these impacts.

The purpose of security risk management is to identify potential software security problems before they occur so that software security risk handling activities can be planned and invoked as needed across the life of the space flight software product or project. 

Risk is incomplete software components and untested software capabilities. Requirements are the basis for a software product. They identify the needs to be addressed, the behavior of the system, the desired quality of the software, and the constraints under which the problem is to be solved. They specify the product the provider is to deliver to a customer.  Requirements serve as the basis for verification activities allowing the developing organization and the customer to judge the completeness of the product.

Risk is that all technical authorities have not agree with the risk associated with any tailored NPR 7150.2 requirements. incomplete implementation of requirements. Software requirements tailoring is the process used to seek relief from NPR requirements consistent with program or project objectives, acceptable risk, and constraints. To accommodate the wide variety of software systems and subsystems, application of these requirements to specific software development efforts may be tailored where justified and approved. To effectively maintain control over the application of requirements in this directive and to ensure proposed tailoring from specific requirements are appropriately mitigated, NASA established TA governance. 

Requirements are the basis for a software product. They identify the needs to be addressed, the behavior of the system, the desired quality of the software, and the constraints under which the problem is to be solved. They specify the product the provider is to deliver to a customer.  Requirements serve as the basis for verification activities allowing the developing organization and the customer to judge the completeness of the product. Missing details in the software requirements, # of estimated SLOC to be developed by the project/ # of detailed software requirements is greater than 50 or the number of TBD/TBC/TBRs in the software requirements document exceeds 10.

Limited or no control of the software development and test schedule, limited insight into the complete set of required software functionality, missing project schedule milestones

Inability to monitor key software observables (e.g. number of failed login attempts, performance changes, internal communication changes) to detect adversarial actions that threaten mission success. Monitoring of key software observables (e.g., number of failed login attempts, performance changes, internal communication changes) is needed to detect adversarial actions that threaten mission success. When an adversarial action occurs, it should be reported. Raw event data should be further analyzed to determine whether an anomalous event represents an attack and if so, the nature of the attack.

Fault management requirements are the basis for a software product fault detection, isolation and recovery activities. They identify the fault needs to be addressed, the behavior of the system during a fault situation, the desired quality of the software, and the constraints under which the fault is to be solved. They specify the product the provider is to deliver to a customer.  Requirements serve as the basis for verification activities allowing the developing organization and the customer to judge the completeness of the product.

Risk is incorrect sensor reading and premature fault indications

OTS Software testing is required to ensure that the OTS software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations. Testing of embedded COTS, GOTS, MOTS, Open Source Software (OSS), or reused software component is required to be at the same level required to accept a custom-developed software component for its intended use.

The principle risk is miscommunication between engineering teams about the meaning of data conveyed across major public interfaces. It's not enough to specify semantics in an interface control document and rely on human inspection. This principle presumes that elements developed by an individual programmer, or within a cohesive team are more likely to be self-consistent, whereas external interfaces are less likely to be consistent. Data quality indicators are for operators or software to recognize unreliable data so that actions should not be taken as a response to displayed data.  With missing quality indicators, Operators or software may act on unreliable data and respond inappropriately thus affecting system reliability 

OTS Requirements are the basis for a software product. They identify the needs to be addressed, the behavior of the system, the desired quality of the software, and the constraints under which the problem is to be solved. They specify the product the provider is to deliver to a customer.  OTS Requirements serve as the basis for verification activities allowing the developing organization and the customer to judge the completeness of the product. 

Bidirectional traceability matrices help ensure that all the required software requirements (only what is required is developed) are addressed in software design and tested in the software testing activities. Bidirectional traceability matrices also make it less likely that requirements are misinterpreted as they are refined.

Risk is incomplete software components and untested software capabilities. Requirements are the basis for a software product. They identify the needs to be addressed, the behavior of the system, the desired quality of the software, and the constraints under which the problem is to be solved. They specify the product the provider is to deliver to a customer.  Requirements serve as the basis for verification activities allowing the developing organization and the customer to judge the completeness of the product.

Risk is incomplete software components and untested software capabilities. Requirements are the basis for a software product. They identify the needs to be addressed, the behavior of the system, the desired quality of the software, and the constraints under which the problem is to be solved. They specify the product the provider is to deliver to a customer.  Requirements serve as the basis for verification activities allowing the developing organization and the customer to judge the completeness of the product.

Risk is incomplete or missing software requirements, insufficent requirements can lead to software functionality not being tested.

Risk is incomplete or missing software requirements, insufficent requirements can lead to software functionality not being tested.

Regression testing calls for Identification of a set of tests that can show that critical functions still perform as expected. These regression tests should be rerun after any changes are made in the system to ensure that the critical functions still perform correctly following the changes made to the system.

Cybersecurity testing is done to ensure that the mitigations put into place to address any cybersecurity weaknesses or vulnerabilities work properly, and allow the system to operate as planned with no security breaches causing unexpected and potentially hazardous operational conditions.

Risk is damage to the system and or test delays due to software bugs. Software testing is required to ensure that the software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations.

Ensuring the Software Test Plan follows a template and includes specified information ensures consistency of test plans across projects, ensures proper planning occurs, and prevents repeating problems of the past. Planning the software testing activity allows thorough deliberation of tasks, methods, environments, and related criteria before they are implemented. Planning also allows the project team to improve a current project, based on lessons learned from previous projects, including using more appropriate or efficient techniques and ensuring the performance of steps previously missed or not included. As with any task, having a plan in place ensures that all necessary and required tasks are performed. Development of that plan provides the opportunity for stakeholders to give input and assist with the documentation and tailoring of the planned testing activities to ensure the outcome will meet the expectations and goals of the project.

Validation is a process of evaluating work products to ensure that the right behaviors have been built into the work products. The right behaviors adequately describe what the system is supposed to do and what the system is supposed to do under adverse conditions. They may also describe what the system is not supposed to do. Validation is performed to assure that the specified software systems fulfill their intended use when placed on the targeted platform in the target environment (or simulated target environment). The methods used to accomplish validation on the actual target platform or in a high fidelity simulator may include aspects that were applied to previous software work products (requirements, designs, prototypes, etc.). The use of these methods provides continuity of results through the assembling system. The use of the high-fidelity or targeted system allows the software developers to check systems-level interfaces, memory performance and constraints, event timing, and other characteristics that can only be evaluated properly in the real system or near-system environment. Validation activities include preparation, performance, analysis of results, and identification of corrective action. Validation at the systems level ensures that the correct product has been built.

Incomplete software testing. Software testing is required to ensure that the software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations.

Incomplete software testing. Software testing is required to ensure that the software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations.

Incomplete software testing or software defects because software was tested to a simulation in place of the real hardware. Software testing is required to ensure that the software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations.

Incomplete software testing. Independent software testing is a software testing approach that employs rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.), and off-nominal conditions (response to faults, responses to hazardous conditions, etc.). The goal of the independent software testing effort is to contribute to the assurance conclusions to the project and stakeholders based on evidence found in software development artifacts and risks associated with the intended behaviors of the software

Incomplete software testing or software defects because software was tested using engineering hardware in place of the real flight hardware. Software testing is required to ensure that the software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations.

Incomplete software testing and missing software defects. Test results are the basis for confirming that the team has fulfilled the software requirements in the resulting software product. To make such decisions, test results must be reviewed and evaluated using a documented, repeatable process. The team can derive quality conclusions by capturing the actual test results, comparing them to expected results, analyzing those results against pre-established criteria, and documenting that analysis/evaluation process. Software testing is required to ensure that the software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations.

Risk is errors in the flight software development tools create errors and defects in the flight software product. Performing verification and validation (V&V) to accredit software models, simulations, and analysis tools is important to ensure the credibility of the results produced by those tools. Critical decisions may be made, at least in part, based on the results produced by models, simulations, and analysis tools. Reducing the risk associated with these decisions is one reason to use accredited tools that have been properly verified and validated. Performing V&V to approve Software models, simulations, analysis tools, and software tools is important to ensure the credibility of those tools’ results and how the tool directly affects the Software being developed. Critical decisions may be made, at least in part, based on the results produced by these tools. Reducing the risk associated with these decisions is one reason to assure that the approved tools have been properly verified and validated. Information regarding specific 

Risk is unclear guidance on milestone reviews and final software product capability, functionality, and testing.  Acceptance criteria is an important component. It clearly defines the scope, desired outcomes of, and testing criteria for pieces of functionality that the delivery team is working on. The process of creating and agreeing on acceptance criteria itself is also an invaluable communication opportunity between developers and product.

Risk is unclear guidance on software product capability, functionality, and testing.  Acceptance software test criteria is an important component. It clearly defines the scope, desired outcomes of, and testing criteria for pieces of functionality that the delivery team is working on. The process of creating and agreeing on acceptance criteria itself is also an invaluable communication opportunity between developers and product.

Risk is unclear guidance on milestone reviews and final software product capability, functionality, and testing.  Acceptance criteria is an important component. It clearly defines the scope, desired outcomes of, and testing criteria for pieces of functionality that the delivery team is working on. The process of creating and agreeing on acceptance criteria itself is also an invaluable communication opportunity between developers and product.

Incomplete software testing or software defects because software commands were not tested. Software command testing is required to ensure that the software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations.

Incomplete software testing or software defects because software data inputs were not tested. Software data input testing is required to ensure that the software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations.

Incomplete software testing or software defects because software commands for updated and update capabilities were not tested. Software commands for updated and update capabilities testing is required to ensure that the software meets the agreed requirements and design. The application works as expected. The application doesn’t contain serious bugs, and the software meets its intended use as per user expectations.

The risk is that the software will crash in conditions of insufficient computational resources (such as memory or disk space), unusually high concurrency, or denial of service attacks.

Risk is incomplete software components and untested software capabilities. Requirements are the basis for a software product. They identify the needs to be addressed, the behavior of the system, the desired quality of the software, and the constraints under which the problem is to be solved. They specify the product the provider is to deliver to a customer.  Requirements serve as the basis for verification activities allowing the developing organization and the customer to judge the completeness of the product.

Risk is that the software development does not have the necessary skills and processes in place to produce reliable products within cost and schedule estimates. 

Risk not enough time in the schedule to perform software development and testing activities. 

Risk of not performing IV&V is potential software defects and a lack of rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.) and off-nominal conditions (response to faults, responses to hazardous conditions, etc.).IV&V is a technical discipline of software assurance that employs rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.), and off-nominal conditions (response to faults, responses to hazardous conditions, etc.). The goal of the IV&V effort is to contribute to the assurance conclusions to the project and stakeholders based on evidence found in software development artifacts and risks associated with the intended behaviors of the software.

NASA requires electronic access to software source code developed by suppliers with NASA funding to enable independent evaluations, checks, reviews, and testing.  Source code is also needed to be able to run static code analysis on the product to check for software errors, security issues, and coding standard compliance. The electronic availability of the software work products, and associated process information, facilitates post-delivery testing that is necessary for assessing as-built work product quality, and for the porting of products to the appropriate hosts. This access also accommodates the longer-term needs for performing maintenance, assessing operation or system errors, addressing hardware and software workarounds, and allowing for the potential reuse of the software on future NASA projects.

Risk of not performing software assurance and IV&V is potential software defects and a lack of rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.) and off-nominal conditions (response to faults, responses to hazardous conditions, etc.). Software assurance and IV&V are technical disciplines of software assurance that employs rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle. The evaluation of products and processes throughout the life cycle demonstrates whether the software is fit for nominal operations (required functionality, safety, dependability, etc.), and off-nominal conditions (response to faults, responses to hazardous conditions, etc.). The goal of the software assurance and IV&V efforts is to contribute to the assurance conclusions to the project and stakeholders based on evidence found in software development artifacts and risks associated with the intended behaviors of the software.

NASA projects demonstrate the following three key reasons for software measurement activities: to understand and model software engineering processes and products, to aid in assessing the status of software projects, and to guide improvements in software engineering processes. What gets measured gets managed.

Baseline set of requirements, software engineering practices and software assurance practices to reduce software engineering risks on NASA projects and programs. 

Risk is missing or incorrectly flowing down a baseline set of requirements, software engineering practices and software assurance practices to reduce software engineering risks on NASA projects and programs. 

Risk is that the software development does not have the necessary skills and processes in place to produce reliable products within cost and schedule estimates. 

Risk is incompatible software, understanding the functionality of the software code, retesting the software in the new environment, maintenance issues, and legal issues

Risk is incomplete software components and untested software capabilities. Requirements are the basis for a software product. They identify the needs to be addressed, the behavior of the system, the desired quality of the software, and the constraints under which the problem is to be solved. They specify the product the provider is to deliver to a customer.  Requirements serve as the basis for verification activities allowing the developing organization and the customer to judge the completeness of the product.

Missing insurance that the processes, procedures, and products used to produce and sustain the software conform to all specified requirements and standards that govern those processes, procedures, and products, Missing ability to determine the degree of software quality obtained by the software products. Inability to ensure that the software systems are safe and that the software safety-critical requirements are followed and that the software products are secure.  Missing employing rigorous analysis and testing methodologies to identify objective evidence and conclusions to provide an independent assessment of critical products and processes throughout the life cycle.

Risk is that the software development does not have the necessary skills and processes in place to produce reliable products within cost and schedule estimates. 

Risk is that the software development does not have the necessary people in place, skills and processes in place to produce reliable products within cost and schedule estimates. 

Risk is that the software development does not have the necessary people in place, skills and processes in place to produce reliable products within cost and schedule estimates. 

Resource oversubscription is a severe fault condition that can lead to unpredictable behavior of the software system and render it inoperable. Examples of these situations include buffers overflowing, exceeding a rate group time boundary, and excessive inputs or interrupts. Software should be designed to provide easy and timely visibility into the use of computing resources during testing and operations.

Risk is that the software development does not have the necessary skills and processes in place to produce reliable products within cost and schedule estimates. 

Risk is that the software development does not have the necessary skills and processes in place to produce reliable products within cost and schedule estimates. The staffing plan should provide for filling key software roles early in the project or task life-cycle and provide for retention of a cadre of experienced development and test personnel through delivery and operations.

Risk is that the software development team does not have the necessary hardware interface data and experience in place to produce reliable products within cost and schedule estimates. Risk associated with developing software at the same time as the hardware interfaces are being developed or defined, risk of misunderstanding the software hardware interfaces and not having the hardware interfaces available to use in the software development.

Resource oversubscription is a severe fault condition that can lead to unpredictable behavior of the software system and render it inoperable. Examples of these situations include buffers overflowing, exceeding a rate group time boundary, and excessive inputs or interrupts.

Risk is that the software development does not have the necessary skills and processes in place to produce reliable products within cost and schedule estimates.