Consult Center Process Asset Libraries (PALs) for Center-specific guidance and resources related to software assurance.

Software quality assurance implementation is a balancing activity that must be tailored as project appropriate.

"No project in the history of Software Development at NASA has had 'enough' money, especially when it comes to implementing Software Quality Assurance activities. It is not possible to achieve all aspects of quality because of interrelationships. Software Quality Assurance engineers must determine, based on experience, what trade-offs are to be made within the specific project since each project has different objectives. Gilles defined some of these relationships between the quality assurance criteria e.g., an inverse relationship between usability and efficiency; a direct relationship between maintainability and reusability ². To make the best trade-offs, all relationships must be understood and experience used to interpret the impact." (Lesson 2 of 12)

Software quality assurance (SQA) must evaluate the process as well as the products.

"Within NASA, SQA tends to focus on the products produced, such as the requirements documents, design, code listing, and test plans. But the focus of SQA is to monitor continuously throughout the Software Development Life Cycle to ensure the quality of the delivered products. This requires monitoring both the processes and the products. In process assurance, SQA provides management with objective feedback regarding compliance to approved plans, procedures, standards, and analyses. Product assurance activities focus on the changing level of product quality within each phase of the life cycle, such as the requirements, design, code, and test plan. The objective is to identify and eliminate defects throughout the life cycle as early as possible, thus reducing test and maintenance costs." (Lesson 3 of 12)

There must be a software assurance plan.

"Most project managers feel they have too many plans, and the suggestion of another one for SQA might be the proverbial straw that breaks the camel's back! But the success of any undertaking is to know what one is trying to achieve and how one expects to accomplish it, hence, the necessity of a plan for SQA. The plan will specify the goals, what is to be performed, standards against which the development work is to be measured, all relevant procedures, and the organizational structure of the Quality Assurance within the project. At NASA a software assurance plan is required." (Lesson 4 of 12)

Software quality assurance must span the entire software development life cycle.

"Software Development starts at the concept phase and continues through maintenance. SQA activities and funding should also start at the concept definition and continue through the entire life cycle. At each phase, there are activities that could be performed ⁴. The project and the quality assurance office must work together to negotiate what activities are appropriate at each phase based on risks and resources. The Quality Assurance Plan should indicate what activities will be performed, the decision basis, and the trade-offs made." (Lesson 5 of 12)

Requirements are the birthplace of successful projects.

"Although SQA is performed across the entire life cycle, success of a project can often be determined by the attention paid to requirements. The earlier in the life cycle potential risks are identified, the easier it is to eliminate or at least manage the conditions that introduce that risk. Problems not found until the testing phase are approximately 14 times more costly to fix than if found and fixed earlier in the requirements phase. The first tangible representation of the capabilities produced is the requirements specification document, be they system, hardware, software, or operational requirements. The document also serves to establish the basis for all the project's engineering management and assurance functions. If the quality of the requirements specification is poor, the project is at risk even before work begins.⁷ Therefore, a specific lesson in SQA is the emphasis on requirements." (Lesson 6 of 12)

Software quality assurance does not equal testing.

"Too often project managers assume they have Quality Assurance since they are doing testing already, or believe they don't need quality assurance until testing. These are incorrect assumptions based on lack of experience, understanding and/or knowledge. From the aspect of quality assurance, the purpose of testing is to:

• Assure problems are documented, corrected, and used for process improvement.
• Assure problem reports are assessed for their validity.
• Assure reported problems and their associated corrective actions are implemented in accordance with customer approved solutions.
• Provide feedback to the developer and the user of problem status.
• Provide data for measuring and predicting Software Quality and Software Reliability."

(Lesson 7 of 12)

Metrics are a necessity.

"Software Metrics are often ignored during the early Software Development Life Cycle phases and not actively associated with SQA - but should be. For SQA practitioners, with their responsibility for assuring both the processes and the products of the Software Development, measurement is critical. Throughout each of the life cycle phases metrics have proven invaluable in the evaluation of the quality of the products and processes ⁵." (Lesson 8 of 12)

Safety and reliability are critical aspects of SQA.

"Safety is a team effort and everyone's responsibility. Software within NASA is a vital part of the system and can impact the safety of the mission. Project managers, Systems Engineers, Software Leads and engineers, Software Assurance or Quality Assurance, and System Safety Personnel all play a part in creating a safe system. Reliability and safety cannot be tested at the end of a project; they must be built in as the software is being Developed.
Reliability impacts safety - a system cannot be deemed safe if it is not reliable." (Lesson 9 of 12)

Independent verification and validation (IV&V) is an important tool within SQA.

"NASA defines Software IV&V as a Systems Engineering Process employing rigorous methodologies for evaluating the correctness and quality of the software product throughout the Software Development Life Cycle. Without SQA, IV&V is expensive and often not as effective. SQA is a broad 'blanket' across the project, overseeing all process and product activities, including software. IV&V focuses on only those processes and products determined to have the highest risk, doing an in-depth evaluation of them. IV&V should not be used on all projects, but instead as a tool to increase reliability and the probability of mission success! ⁴" (Lesson 10 of 12)

Hardware does not equal software!

"The influence of Hardware Quality Assurance is evident in the Software Quality Assurance practitioner community, where hardware-intensive systems and typical hardware-related concerns predominate. Two issues which dominate discussions about hardware are time and operating conditions. Software, however, is built with different constraints and considerations. NASA has grappled with these differences and the best approach for recognizing the differences (yet similarities) of hardware and software quality assurance. At Goddard Space Flight Center (GSFC), and within other NASA Centers, hardware and software QA are in one department. This allows for them to work together, since both are needed for a successful project, yet are still separate in other areas where needed." (Lesson 11 of 12)

Risk Management is not optional.

"Risk is a daily reality on all projects, and Continuous Risk Management should become just as routine. It should be ongoing and comfortable and neither imposed nor forgotten. Like any good habit, it should seamlessly fit into the daily work.
Software Risk Management is important because it helps avoid disasters, rework, and overkill. The objectives of Software Risk Management are to identify, address, and eliminate software risk items before they become threats to success or major sources of rework. Good project managers are also good managers of risk. It makes good business sense for all software development projects to incorporate risk management as part of project management." (Lesson 12 of 12)