Modern static code analysis tools can identify a variety of issues and problems, including but not limited to dead code, non-compliances with coding standards, security vulnerabilities, race conditions, memory leaks, and redundant code. Typically, static analysis tools are used to help verify adherence with coding methods, standards, and/or criteria. While false positives are an acknowledged shortcoming of static analysis tools, users can calibrate, tune, and filter results to make effective use of these tools. Software peer reviews/inspections of code items can include reviewing the results from static code analysis tools.
Static analysis tools may not be readily available for some platforms or computing languages. If static analysis tools are determined to be unavailable, the project can document the alternate manual methods and procedures to be used to verify and validate the software code. These manual methods and procedures will be addressed or referenced in the project's compliance matrix against this requirement.
For critical code, it is essential to use sound and complete static analyzers. Sound and complete analyzers guarantee that all errors are flagged and that no false negatives (i.e., an erroneous operation being classified as safe) are generated. Such commercial analyzers are expensive but necessary for critical applications. Note that sound and complete static analyzers are now available free of charge for C and C++ software systems (see the Tools Table provided under the tab).
The cost of sound and complete commercial tools can be prohibitive for projects working on non-critical software systems. However there are other static analyzers available free of charge, which can identify many common errors or deviations from standards or good coding practices. It is a best practice to use some type of static analysis in all software development efforts, but required for mission-critical software.
Introducing the routine use of static analyzers in an existing development process can be a tricky proposition. Even if the software team is convinced of the benefits such tools can bring, projects need to be careful and make the introduction of static analysis as unobtrusive as possible. Fortunately, current generation static analyzers come with interfaces similar to compilers, such as GNU (Unix-like computer operating system developed by the GNU Project) Compiler Collection (GCC), and/or Integrated Development Environments (IDEs), such as Eclipse, which helps in gaining acceptance. However, the user interface is not the only factor to consider. Here is a non-exhaustive list of potential challenges:
- The static analyzer may not accept your code. It is not uncommon that the first problem (after installation) with a static analyzer is with its inability to process code.
Today, most commercial static analyzers rely on the same front-end code processor (e.g., the Edison Design Group (EDG) front-end for C and C++), which means that they in principle are able to process the same code. However, the algorithms used for the analysis might not handle code constructs in similar manners, and therefore, there are differences amongst tools. This problem is even more prevalent with tools that are free of charge because they may rely on different parsers, so commercial tools are recommended.
Guidance: For existing projects interested in evaluating commercial static analyzers, consider the following pointers:
- Start with a static analyzer that comes with a free evaluation period and evaluate it on your most “tricky” code, i.e., with all the language constructs and libraries that the team commonly uses.
- Choose a commercial analyzer based your evaluation, not on a vendor demonstration; vendors will only show what works, not the potential challenges.
- Choose a tool that comes with extensive technical support. Your application might have quirks that do not seem like quirks to your project, but might be to others. So, readily available support can be a key factor in effectively using a static analyzer. For this reason, commercial tools are recommended. Free static analyzers will not generally provide the support the software team may need.
- Choose a tool that relies on commonly used parsers to mitigate future issues. For example, for C and C++, it is best to choose a tool that relies on GCC for processing code. In general, GCC can process all C and C++ programs and also offers options to cover old constructs or customized extensions.
Matching a static analyzer with the development team’s compiler will eliminate many challenges. Even if the tool has a good parser, make sure to give the tool access to the right libraries. Remember that a static analyzer is essentially a fancy compiler that does not produce executable code. Getting the compiling and linking environment right can be challenging if the chosen tool does not use the same parser as the team’s compiler.
- The static analyzer may take an extensive amount of time to finish the analysis. In cases where the analysis is in-depth and code is “complex”, a static analyzer can take an inordinate amount of time to come back with results. Using a static analyzer includes making configuration decisions regarding precision, scalability, and language construct coverage. The most precise analyzers use sophisticated algorithms and usually have large memory requirements. These factors may result in slow analysis times and in running out of memory (which causes swapping with “slow” storage mechanisms, e.g., hard disks).
Guidance: Select and configure a static analyzer to operate within a diurnal development cycle and consider two analysis steps. As a rule of thumb, don’t set up a static analysis session to take more than eight hours. This will ensure that you can run overnight and offer new results every morning. Note that the most precise static analyzers come with different levels of analysis, which can be configured to fit time requirements and later to run a precise analysis of the project at select points in the development process. Fast but imprecise analyzers can be added to produce quick results, but will produce more warnings. Care must be taken to filter results, remove irrelevant findings, and also to configure the tool without suppressing useful, if noisy, code checks.
With static analysis, there is often a trade between precision and scalability. You can expect to spend time upfront configuring the static analyzer to the particular needs of the project. For example, a quick, but crude, static analyzer will produce quick feedback within a normal work cycle, and, a slow, but precise, analyzer can be run before a major release. Note that there are some precise commercial tools that are customizable for rapid analysis.
- The static analyzer comes back with too many warnings. Fortunately, there are now many mechanisms (filtering, analysis tuning, increased precisions) to deal with this problem. This challenge also stems from the need to trade precision for scalability. For example, you may configure an analyzer to be complete, which means that it needs to flag every possible error. In doing so, it relies on approximations (called abstractions in the literature), which lead the analysis to consider execution paths that may not be realizable. Therefore the analyzer may not distinguish between a real error and a possible error (warnings). The question is: “What can be done when faced with an overwhelming number of warnings?”
Guidance: First, make sure you are familiar with the static analyzer. Static analyzers that look for errors fall into two categories: debuggers and certifiers. Debuggers try to find bugs quickly, and in doing so, they cut many corners; thus they usually generate many warnings. Certifiers are slower but they try to characterize every operation as safe, unsafe, potentially unsafe, or dead. Depending on your need, pick one or the other, or use one or the other at the proper place in your process. Note that certifiers are recommended for safety critical code.
Second, make sure that your static analyzer has a filtering capability. Most come with a means of filtering by types of errors. However, if possible, pick a tool that can rank warnings so that you can focus on the most likely bugs. Some tools will do that automatically and present only a percentage of all the warnings. Lastly, it may be necessary to write a customized script to filter results.
When triaging warnings or errors, it is recommended to first process errors that appear early in the execution and re-run the static analyzer to avoid a lot of unnecessary work. As with compiler errors, fixing an early error might lead to the elimination of many later errors/warnings. This dependency between warnings is why it is recommended to run analysis during the development process; the results are addressed and cascading side effects less likely to accumulate.
Checking coding standards/criteria through the use of a static analysis tool complements manual code reviews (SWE-087), and results in safer software systems. Coding standards/criteria (see also SWE-061) ensures code consistency across a project, and facilitates not only reviews but also software system integration. Software components that do not comply with coding standards can result in incorrect assumptions about the component, which can result in errors at later development stages. Debugging can also be hampered by deviations from coding standards/criteria since they can result in misleading assumptions. Enforcement of coding standard yields benefits that can continue into the later software life-cycle phases, reducing software maintenance complexity and fostering code reuse.
Static analysis is applicable to a wide range of software systems, with exceptions when tools are not available for a particular platform or programming language. Note that there are many available tools for widely used languages such as Ada, C, C++ and Java (see the Tools Table referenced under the tab).
The NASA IV&V Program maintains access to an extensive set of static code analyzers to support the evaluation of a diverse set of coding languages including C, C++, Ada, Java, and Fortran. IV&V has established a robust infrastructure to support the independent, meaning outside of the developers software development environment (SDE), execution of these tools as well as a mature capability for the prioritization and evaluation of the analyzer results. The IV&V Program regularly utilizes these analyzers in the evaluation of the Agency’s safety and mission critical software and can provide fee for service support for projects not currently receiving IV&V from the NASA IV&V Program. For more information please contact the IV&V Program business representatives via the IV&V Program website .