1. Requirements

3.7.5 If a project has safety-critical software, the project manager shall ensure all identified safety-critical software components have a cyclomatic complexity value of 15 or lower. Any exceedance shall be reviewed and waived with rationale by the project manager or technical approval authority.

1.1 Notes

Cyclomatic complexity is a metric used to measure the complexity of a software program. This metric measures independent paths through the source code. The point of the requirement is to minimize risk, minimize testing, and increase reliability associated with safety-critical software code components, thus reducing the chance of software failure during a hazardous event. The software developer should assess all software safety-critical components with a cyclomatic complexity score over 15 for testability, maintainability, and code quality. For more guidance on this requirement, see NASA-HDBK-2203.

1.2 History

1.3 Applicability Across Classes

Key:    - Applicable | - Not Applicable

2. Rationale

To minimize risk, ensure complete software testing and increase reliability associated with safety-critical software code components. 

3. Guidance

3.1 Ensure that all identified safety-critical software components have a cyclomatic complexity value of 15 or lower. 

The requirement is to minimize risk, minimize testing, and increase reliability associated with safety-critical software code components, thus reducing the chance of software failure during a hazardous event. A section of source code's cyclomatic complexity is the number of linearly independent paths within it—where "linearly independent" means that each path has at least one edge that is not in one of the other paths. For instance, if the source code contained no control flow statements (conditionals or decision points), the complexity would be 1 since there would be only a single path through the code. If the code had one single-condition IF statement, there would be two paths through the code: one where the IF statement evaluates TRUE and another one where it evaluates to FALSE so that the complexity would be 2. Two nested single-condition IFs, or one IF with two conditions, would produce a complexity of 3.

Purpose:

• Limiting complexity during development
• Measuring the "structuredness" of a program - concerned with determining what the control flow graph
• Implications for software testing - Another application of cyclomatic complexity is determining the number of test cases necessary to achieve thorough test coverage of a particular module.
• Correlation to the number of defects - Some studies find a positive correlation between cyclomatic complexity and defects: functions and methods with the highest complexity tend to contain the most defects. However, the correlation between cyclomatic complexity and program size (typically measured in lines of code) has been demonstrated many times.  Studies that controlled for program size (i.e., comparing modules with different complexities but similar size) are generally less conclusive, with many finding no significant correlation, while others find the correlation. Some researchers who have studied the area question the validity of the studies' methods finding no correlation.
• Code maintainability 
  
• Reduces the coupling of code. The higher the cyclomatic complexity number, the more coupled the code is. Highly coupled code cannot be modified easily and independently of other code.
• Ease of understanding the code increases as the complexity decreases. With a higher complexity number, the programmer has to deal with more control paths in the code, which leads to more unexpected results and defects.
• Ease of testing. If a method has a cyclomatic complexity of 10, it means there are 10 independent paths through the method. This implies that at least 10 test cases are needed to test all the different paths through the code. The lesser the number, the easier it is to test.

See also SWE-023 - Software Safety-Critical Requirements and SWE-134 - Safety-Critical Software Design Requirements. SWE-135 - Static Analysis, 

3.2 Software static analysis tools can provide the cyclomatic complexity value of any software component.  

3.2.1 What is Cyclomatic Complexity?

Cyclomatic complexity is a software metric used to measure the complexity of a program. These metrics measure independent paths through program source code. An independent path is defined as a path with at least one edge that has not been traversed before in any other path. Cyclomatic complexity can be calculated concerning functions, modules, methods, or classes within a program.

Thomas J. McCabe developed this metric in 1976, and it is based on a control flow representation of the program. Control flow depicts a program as a graph that consists of Nodes and Edges.

In the graph, nodes represent processing tasks while edges represent control flow between the nodes.

3.2.2 Flow graph notation for a program:

Flow Graph notation for a program defines several nodes connected through the edges. Below are Flow diagrams for statements like if-else, While, until, and normal sequence of flow.

Cyclomatic complexity is a software metric used to measure the complexity of a program. These metrics measure independent paths through program source code. An independent path is defined as a path with at least one edge that has not been traversed before in any other path.  It quantitatively measures the number of linearly independent paths through a program's source code. The application of the requirement is to limit the complexity of routines during program development; programmers should count the complexity of the modules they are developing and split them into smaller modules whenever the cyclomatic complexity of the module exceeded 15. The NIST Structured Testing methodology adopted this practice. The figure of 10 had received substantial corroborating evidence.  There are occasional reasons for going beyond the agreed-upon limit. It phrased its recommendation as "For each module, either limit cyclomatic complexity to 15 or provide a written explanation of why the limit was exceeded. For example, if the source code contains no control flow statement, its cyclomatic complexity will be 1, and the source code contains a single path in it. Similarly, if the source code contains one if condition, then cyclomatic complexity is 2 because there will be two paths: true and the other for false.

Several studies have investigated the correlation between cyclomatic complexity numbers with the frequency of defects occurring in a function or method. Studies have found a positive correlation between cyclomatic complexity and defects: functions and methods with the highest complexity also tend to contain the most defects. However, international safety standards like ISO 26262  ³⁷⁶  and IEC 62304  ³⁷⁵  mandate coding guidelines that enforce low code complexity.

3.2.3 Use of Cyclomatic Complexity:

• Limit code complexity.
• Determine the number of test cases required.
• Determining the independent path executions has thus proven to be very helpful for Developers and Testers.
• It can make sure that every path has been tested at least once.
• This helps to focus more on uncovered paths.
• Code coverage can be improved.
• The risk associated with the program can be evaluated.
• These metrics being used earlier in the program help in reducing the risks.

Higher numbers of cyclomatic complexity are bad, and lower cyclomatic complexity numbers are good. That's because code with high complexity is difficult to test. And it's likely to result in errors. So, code with low complexity is easier to test. And it's less likely to produce errors. 

The following table gives an overview of the complexity number and corresponding meaning:

For safety-critical code, anything over 15 should be assessed for testability, maintainability, and code quality.

If the software safety-critical components have a cyclomatic complexity value of 16 or higher, then work with software assurance to provide a risk assessment showing why the cyclomatic complexity value needs to be higher than ten and why the software component cannot be structured to be 15 or lower.

Additional information can be found in NASA/TM−20205011566, NESC-RP-20-01515, Cyclomatic Complexity, and Basis Path Testing Study  ³⁷⁷.  See also 7.21 - Multi-condition Software Requirements

3.2.4 There are different types of cyclomatic complexity.

For example, standard cyclomatic complexity (CC or CC1) is equal to the number of decisions + 1 for the code in question.

Strict cyclomatic complexity (CC2) adds one for each Boolean condition within a compound decision predicate, rendering CC2 difficult to “game” (i.e., lowering the measured complexity through moving otherwise nested code decision blocks into complex multi-condition decision logic).

Another variation, modified cyclomatic complexity (CC3), reduces the penalty on multi-way decision branches, such as switch() statements in the C language, by counting them as one decision. CC3 may be sensible for certain non-safety-critical application types that use event handlers or finite state machines heavily.

3.3 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook:

3.4 Center Process Asset Libraries

See the following link(s) in SPAN for process assets from contributing Centers (NASA Only). 

4. Small Projects

No additional guidance is available for small projects.

5. Resources

5.1 References

5.2 Tools

6. Lessons Learned

There are currently no Lessons Learned identified for this requirement.

7. Software Assurance

7.1 Tasking for Software Assurance

7.2 Software Assurance Products.

• Software assurance status reports
• Software hazard reports
• Software design analysis
• Source code quality analysis
• Software complexity data
• Software assurance risk assessment of any software developers' rationale if requirements are not met.

  
  

  Objective Evidence

  • Evidence of confirmation that all identified safety-critical software components have a cyclomatic complexity value of 15 or lower or provide a risk assessment explaining why the cyclomatic complexity value needs to be higher than 15 and why the software component cannot be structured to be lower than 15.
  • NPR 7150.2 and NASA-STD-8739.8 requirements mapping matrices signed by the engineering and SMA technical authorities for each development organization.
   Definition of objective evidence

  Objective evidence is an unbiased, documented fact showing that an activity was confirmed or performed by the software assurance/safety person(s). The evidence for confirmation of the activity can take any number of different forms, depending on the activity in the task. Examples are:

  • Observations, findings, issues, risks found by the SA/safety person and may be expressed in an audit or checklist record, email, memo or entry into a tracking system (e.g. Risk Log).
  • Meeting minutes with attendance lists or SA meeting notes or assessments of the activities and recorded in the project repository.
  • Status report, email or memo containing statements that confirmation has been performed with date (a checklist of confirmations could be used to record when each confirmation has been done!).
  • Signatures on SA reviewed or witnessed products or activities, or
  • Status report, email or memo containing a short summary of information gained by performing the activity. Some examples of using a “short summary” as objective evidence of a confirmation are:
    • To confirm that: “IV&V Program Execution exists”, the summary might be: IV&V Plan is in draft state. It is expected to be complete by (some date).
    • To confirm that: “Traceability between software requirements and hazards with SW contributions exists”, the summary might be x% of the hazards with software contributions are traced to the requirements.
  • The specific products listed in the Introduction of 8.16 are also objective evidence as well as the examples listed above.

7.3 Metrics 

• Software cyclomatic complexity # for all identified safety-critical software components

7.4 Guidance 

Confirm that all identified safety-critical software components have a cyclomatic complexity value of 15 or lower. If not, assure that software developers provide a risk assessment explaining why the cyclomatic complexity value needs to be higher than 15 and why the software component cannot be structured to be lower than 15.

The requirement is to minimize risk, minimize testing, and increase reliability associated with safety-critical software code components, thus reducing the chance of software failure during a hazardous event. A section of source code's cyclomatic complexity is the number of linearly independent paths within it—where "linearly independent" means that each path has at least one edge that is not in one of the other paths. For instance, if the source code contained no control flow statements (conditionals or decision points), the complexity would be 1 since there would be only a single path through the code. If the code had one single-condition IF statement, there would be two paths through the code: one where the IF statement evaluates TRUE and another one where it evaluates to FALSE so that the complexity would be 2. Two nested single-condition IFs, or one IF with two conditions, would produce a complexity of 3.

Purpose:

• Limiting complexity during development
• Measuring the "structuredness" of a program - concerned with determining what the control flow graph
• Implications for software testing - Another application of cyclomatic complexity is determining the number of test cases necessary to achieve thorough test coverage of a particular module.
• Correlation to the number of defects - Some studies find a positive correlation between cyclomatic complexity and defects: functions and methods with the highest complexity tend to contain the most defects. However, the correlation between cyclomatic complexity and program size (typically measured in lines of code) has been demonstrated many times.  Studies that controlled for program size (i.e., comparing modules with different complexities but similar size) are generally less conclusive, with many finding no significant correlation, while others find the correlation. Some researchers who have studied the area question the validity of the studies' methods finding no correlation.
• Code maintainability 
  
• Reduces the coupling of code. The higher the cyclomatic complexity number, the more coupled the code is. Highly coupled code cannot be modified easily and independently of other code.
• Ease of understanding the code increases as the complexity decreases. With a higher complexity number, the programmer has to deal with more control paths in the code, which leads to more unexpected results and defects.
• Ease of testing. If a method has a cyclomatic complexity of 10, it means there are 10 independent paths through the method. This implies that at least 10 test cases are needed to test all the different paths through the code. The lesser the number, the easier it is to test.

See also SWE-023 - Software Safety-Critical Requirements and SWE-134 - Safety-Critical Software Design Requirements. 

7.4.1 Software static analysis tools can provide the cyclomatic complexity value of any software component.  Work with Engineering and IV&V (If used on the project) to get the cyclomatic complexity value of all of the safety-critical software components. 

7.4.2 What is Cyclomatic Complexity?

Cyclomatic complexity is a software metric used to measure the complexity of a program. These metrics measure independent paths through program source code. An independent path is defined as a path with at least one edge that has not been traversed before in any other path. Cyclomatic complexity can be calculated concerning functions, modules, methods, or classes within a program.

Thomas J. McCabe developed this metric in 1976, and it is based on a control flow representation of the program. Control flow depicts a program as a graph that consists of Nodes and Edges.

In the graph, nodes represent processing tasks while edges represent control flow between the nodes.

Flow graph notation for a program:

Flow Graph notation for a program defines several nodes connected through the edges. Below are Flow diagrams for statements like if-else, While, until, and normal sequence of flow.

Cyclomatic complexity is a software metric used to measure the complexity of a program. These metrics measure independent paths through program source code. An independent path is defined as a path with at least one edge that has not been traversed before in any other path.  It quantitatively measures the number of linearly independent paths through a program's source code. The application of the requirement is to limit the complexity of routines during program development; programmers should count the complexity of the modules they are developing and split them into smaller modules whenever the cyclomatic complexity of the module exceeded 15. The NIST Structured Testing methodology adopted this practice. The figure of 10 had received substantial corroborating evidence.  There are occasional reasons for going beyond the agreed-upon limit. It phrased its recommendation as "For each module, either limit cyclomatic complexity to 15 or provide a written explanation of why the limit was exceeded. For example, if the source code contains no control flow statement, its cyclomatic complexity will be 1, and the source code contains a single path in it. Similarly, if the source code contains one if condition, then cyclomatic complexity is 2 because there will be two paths: true and the other for false.

Several studies have investigated the correlation between cyclomatic complexity numbers with the frequency of defects occurring in a function or method. Studies have found a positive correlation between cyclomatic complexity and defects: functions and methods with the highest complexity also tend to contain the most defects. However, international safety standards like ISO 26262  ³⁷⁶  and IEC 62304  ³⁷⁵  mandate coding guidelines that enforce low code complexity. See also 7.21 - Multi-condition Software Requirements. 

Use of Cyclomatic Complexity:

• Limit code complexity.
• Determine the number of test cases required.
• Determining the independent path executions has thus proven to be very helpful for Developers and Testers.
• It can make sure that every path has been tested at least once.
• This helps to focus more on uncovered paths.
• Code coverage can be improved.
• The risk associated with the program can be evaluated.
• These metrics being used earlier in the program help in reducing the risks.

Higher numbers of cyclomatic complexity are bad, and lower cyclomatic complexity numbers are good. That's because code with high complexity is difficult to test. And it's likely to result in errors. So, code with low complexity is easier to test. And it's less likely to produce errors. 

The following table gives an overview of the complex number and its corresponding meaning:

For safety-critical code, anything over 15 should be assessed for testability, maintainability, and code quality.

If the software safety-critical components have a cyclomatic complexity value of 16 or higher, then work with engineering to provide a risk assessment showing why the cyclomatic complexity value needs to be higher than ten and why the software component cannot be structured to be 15 or lower.

Additional information can be found in NASA/TM−20205011566, NESC-RP-20-01515, Cyclomatic Complexity, and Basis Path Testing Study  ³⁷⁷.  

7.5 Additional Guidance

Additional guidance related to this requirement may be found in the following materials in this Handbook: