1. Introduction

Model-based software development uses a model as the centerpiece of the development process. Software engineers create a model of the system behavior that can then be translated into different languages such as C or Ada by the appropriate code generator. The model is continually refined throughout the development process and can even be executable. For maximum benefit, the generated code should not be modified by software engineers; when changes are desired, the model is revised and code is generated from the revised model.

Both the modeling tool and the code generation tool are part of a toolset that makes up the model-based development system, referred to in this guidance as MBDS.

Efficiency is a key benefit of using MBDS, but there are many considerations to be reviewed when deciding whether MBDS is the right choice for a project. For a list of pros and cons, see the Determining the need for MBDS section of this guidance.

1.2 Purpose

This document addresses guidance for projects that desire to use MBDS. This guidance addresses:

• Identifying the need for MBDS.
• Choosing an appropriate MBDS.
• Practices to avoid.
• Recommended practices.

1.3 Roles

2. Determining the need for MBDS

Before automatically assuming that MBDS is the right choice, it should be determined whether this type of system is appropriate for the project. The following questions should be addressed when making this determination:

1. Why does the project want to use MBDS? What is the benefit to the project?
2. What is the team's experience with software development using MBDS?
3. What are the pros and cons of using MBDS? Are there specific pros and cons to be weighed for this project?
   1. General pros:
      1. Shifts the focus from implementation to design ²⁴⁴ allowing the focus to be on the problem being solved.
      2. Can improve productivity, quality and complexity hiding. ²⁴⁴
      3. Can provide consistent products and lower error rates. ²⁴⁴
      4. Can create optimized efficient implementations. ²²⁶
      5. Helps avoid programmer errors such as typos, pointer problems, etc. ²²⁶
      6. Can enforce programming practices. ²²⁶
      7. Can provide for shared resource protection, memory management, code and data optimization, process and data distribution, bug avoidance, less testing, and reuse. ²²⁶
      8. Can adapt quickly to changes in requirements. ²²⁶
      9. Increases productivity by maximizing compatibility between systems, simplifying the process of design, promoting communication between individuals and teams working on the system. ²⁵³
      10. Helps recognize problems early on and reduce risk.
      11. Reduces iterations in the development cycle, reducing development time and cost.
   2. General cons:
      1. MBDSs are typically not qualified; therefore, there is no guarantee that their output is correct. ¹⁹³
      2. "Code reviews are still necessary for mission-critical applications, but the generated code is often difficult to understand..." ¹⁹³
      3. "Common modeling and programming languages do not allow important requirements to be represented explicitly (e.g., units, coordinate frames, quaternion handedness); consequently, such requirements are generally expressed informally and the generated code is not traced back to these requirements." ¹⁹³
      4. "Writing documentation is tedious and therefore often not completed or kept up to date." ¹⁹³
      5. "Limited code generation possibilities force developers to start manual programming after design. This kills the idea of full code generation and minimizes the productivity benefits." ²⁴⁴
      6. Flaws in parsers or generators can have wide effects downstream. ²²⁶
4. Will the amount of code the project will be developing warrant/justify the use of MBDS?
5. Does the project or Center already have MBDS that can be used or will the project need to purchase, rent, or develop MBDS? What are the pros and cons of each option?
6. How will using MBDS affect the maintenance of the developed code?

3. Choosing MBDS

Once the decision has been made that MBDS is appropriate for a project, the actual system needs to be selected. When choosing a system, the following considerations should be included in the decision-making process:

1. Ensure each MBDS is verified to provide confidence in the products produced.* ²²⁶
2. When project needs require creation of in-house MBDSs, create them in compliance with NPR 7150.2. ²²⁶
3. Consider the suitability of the modeling language for representing the problem. ²⁷⁶
4. Consider the sufficiency of modeling constructs for generating code by answering the following questions: ²⁷⁶
   
   1. How much of the code can the tool generate?
   2. How much will have to be hand-coded?
5. Consider the maturity of MBDSs for generating quality code. ³⁶⁹
   
   1. Look at the MBDS's history/track record.
   2. Determine what, if any, analysis exists to verify the software produced by the MBDS.
6. Consider the MBDS's integration with other project tools, such as debuggers or static analysis tools. ²⁷⁶
7. Consider methodologies used by the MBDS for translating design into code. ²⁷⁶
8. Select MBDS appropriate to the current application. ²⁷⁶
   
   1. Choose the right tool for the development environment.
   2. Choose the right method for the problem to be solved.
   3. Choose tools and methods that will work together.
9. Consider whether the project team has sufficient understanding of the MBDS to be able to use it efficiently.
10. Consider the domain-relevance of the MBDS. ²⁴⁴
    
    1. Choose MBDS that allows focus to be on the model/design, which is the problem, not the code.
    2. Choose MBDS in which the domain knowledge is made explicit for the development team, being captured in the modeling language and its tool support.
11. Consider the language in which the MBDS generates the source code. ²⁷⁶
12. Consider input models required for translators. ³⁶⁹
    
    1. Consider importance to project of being able to use different input models.
13. Consider project needs of generated output. ³⁶⁹
    
    1. Consider how much flexibility is needed to meet project requirements for generated code.
    2. If each mission can impose its own coding standards for the code being generated, then the flexibility of the output of the generator is important.
    3. Consider whether the project has a need for generating multiple forms of code from the same input, such as 1) C/C++ code, 2) a model representation that can be run through a model-checking tool and 3) a model that can be executed in simulation.
14. Consider project needs for coordinated use of the MBDS by development team members, including members in various locations and members responsible for various parts of the model. ²⁴⁵
15. Consider potential for reuse of the MBDS on future projects. ²²⁶
16. Consider whether the project will need to maintain a specific version of the MBDS for as long as the code is in use.
17. Consider whether the company that produced the MBDS will support the system if it is used over the long term.

4. Issues and Pitfalls, Practices to Avoid

As with any development methodology, there are issues associated with MBDS and practices that can result in unintended consequences. To ensure the best outcome for a project using MBDS, the following recommendations should be part of the process of developing associated procedures:

1. Use a model to generate the code and do not simply develop a model in parallel with the code. ²²⁶
2. Don't generate algorithms as part of the design without a requirement to which they can be traced and verified. ²²⁶
3. Beware of customized versions of "reused" MBDS. ²²⁶
   
   1. MBDS which are maintained internally (e.g., project or center) can have customizations and/or bug fixes that may not be documented or that may have side effects.
4. Testing of the model should not replace thorough system testing. ³⁶⁹
   
   1. One advantage of MBDS is moving some of the testing activities earlier in the life cycle. If major problems are found, they can be resolved with less impact on the budget or schedule of the project.
   2. Disadvantages include a reliance on the automatically generated source code (which may not be generated correctly) and the difficulty of knowing how well the model conforms to reality. Interactions between parts of the system may not be evident until the system is operational.
5. Avoid including too much detail in a single state chart model (may obscure overall functionality). ³⁶⁹
6. Avoid including too little detail in a single state chart model (may render the state chart content-less). ³⁶⁹
7. Don't forget to consider the costs of additional training for developers who have never used this type of tool. These costs result in a first time start up cost from a schedule perspective.
8. Don't forget that all developers need to understand the model, not just the resulting code.
9. Don't forget to maintain the MBDS throughout the life of the code's use.
10. Don't forget to consider support costs when choosing MBDS. The company may not support the tool if the use planned is very long. This is similar to COTS pitfalls
11. Don't forget to consider that inefficient code generated by MBDS may not run sufficiently fast or may use greater resources than hand-developed code.

5. Recommended Practices

The following practices, collected from a variety of sources, are recommended practices when using MBDS in a project. These practices should be considered when developing project or center procedures for the use of MBDS:

1. Evaluate OTS MBDSs for best project fit and to ensure produced code is closest to production code as possible (no residual code, dead code, instrumentation, etc.). ²²⁶
   
   1. Including instrumentation, such as internal or intermediate check points, in production code is problematic as that code has the potential for inadvertent execution.
   2. Including instrumentation and then removing it is also problematic as it changes the production code from the tested, verified code.
2. Test the MBDS using a reference model ¹³³, a graphical test model which allows the functionality of the generator to be confirmed as operating correctly. ¹⁶⁰
   
   1. Invalid models should be rejected by the generator while correct models should be translated properly into code.
   2. Correct functioning of the generator can be ensured if the test model and the generated code behave the same when subjected to the same stimulus/test data.
3. Verify the correct translation from design to code for safety-critical software. ²²⁶
4. Consider running tests against parsers, generators, and configuration data as early as possible; finding bugs early avoids potential problems downstream. ²²⁶
5. Conduct evaluation/verification/qualification for every version of the MBDS. ¹⁹³
6. Conduct evaluation/verification/qualification for each project that uses the MBDS. ¹⁹³
7. Repeat evaluation/verification/qualification if MBDS is updated during the project. ¹⁹³
8. Once the MBDS has been thoroughly reviewed and the resulting code validated, review efforts for that project can focus on the designs rather than the implementations. ³⁶⁹
9. If marketing to others that developed code is model-based, ensure a model is used to generate the code and not simply developed in parallel with the code. ²²⁶
10. Ensure the model is the best, clearest it can be since the model is the root / basis of the generated code. ²²⁶
    
    1. Use good modeling principles ²⁵⁰, including:
       
       1. Keep the model as simple as possible
       2. Keep the model semantically well-defined
       3. Keep the model organized and manageable
       4. Use a practical point of view: if you aren't going to use it, don't model it
    2. Structure the model logically and make it architecturally consistent.
    3. Include notations and comments in the model so the purpose of each element/module is clear.
11. Subject the model to the same level of scrutiny and review as source code. ³⁶⁹
    
    1. Include system engineers and software engineers in joint reviews to identify misunderstood or unclear requirements.
12. Separate generated and manual code. ¹³³
    
    1. Use separate folders/directories for generated code.
    2. Include commentary at the beginning of generated code to clearly identify it as generated.
13. Don't check in generated code. ¹³³
    
    1. Code which is 100 percent generated should neither be changed nor checked into the repository.
    2. Code which is 100 percent generated is a disposable good - the important artifacts are the models.
    3. The only time it can make sense to check-in generated code is when, for some reason, it is impossible to integrate the generator run into the build process.
14. Integrate the generator portion of the MBDS into the build process. ¹³³
15. Generate clean code. ¹³³
    
    1. Generated code should satisfy the same quality requirements that apply to manually written code.
    2. Generated code should follow applicable coding styles and use well-established design patterns.
16. When the code is safety-critical, or resides in an unprotected partition with safety-critical code, the automatically generated code should be subjected to the same rigorous inspection, analysis, and test as hand-generated code. ²⁷⁶
17. One of the primary means by which MBDS gain programming efficiency is through the use/reuse of libraries that raise the level of abstraction. This is not unique to MBDS, but seems to be more prevalent.
    
    1. Make an effort to understand what details are being abstracted and make a conscious choice as to whether the team will be comfortable with the level of abstraction provided.
    2. A good practice is to maintain a controlled library of accepted, reusable model elements and limit developers to using only approved libraries.
    3. A mechanism should exist for getting additional model elements into the approved library.
    4. Each project may need to review the library set for suitability for that application. For example, a model element that is acceptable for processing data on the ground may not be acceptable on a real-time flight control system.

Additional guidance related to auto-generated code may be found in the following related requirement in this Handbook:

6. Resources

6.1 Tools