bannerc

A. Introduction

B. Institutional Requirements

C. Project Software Requirements

D. Topics

E. Tools, References, and Terms

F. SPAN
(NASA Only)

7.21 - Multi-condition Software Requirements

 1. Multi-Logic Testing

1.1 – Definitions

  • Unit Testing
    • Testing of individual routines and modules by the developer or an independent tester (ISO/IEC/IEEE 24765).
    • A test of individual programs or modules in order to ensure that there are no analysis or programming errors (ISO/IEC 2382-20).
    • Test of individual hardware or software units or groups of related units. (ISO/IEC/IEEE 24765)

  • Software Verification
    • Confirmation that products properly reflect the requirements specified for them. Verification ensures that “you built it right.” (Source: IEEE 1012)
  • Software Integration Testing -
    • Integration testing examines if the individual components developed and tested separately, interact as expected when they are combined to form a part of a larger system. (Source: IEEE)
  • Integration Testing -
    • Integration testing examines if the individual components developed and tested separately, interact as expected when they are combined to form a part of a larger system. (Source: IEEE)


The recommendations in this topic are for functional software requirements

  • Although similarities with methods used to ensure code is fully unit tested, code unit testing is not the subject of these recommendations
  • These recommendations are solely about ensuring software requirements are fully verified through test

1.2 Multi-Logic Condition Requirements

  • Simplified Example of Multi-logic Condition Software Requirement:
    • “ If mode is STANDBY and status is (ENABLED or ACTIVE), the software shall store the current time.”

  • For software requirements, there should be very few multi-logic condition requirements
    • Multi-logic condition requirements complicate testing and should be reduced or eliminated, if possible
  • Operators used in textual requirements are typically “and” or “or”

  • The example above can be restated as: If A and (B or C) then store the current time.
    • If the result of “A and (B or C)” is TRUE, then the current time will be stored.
    • Operators are “and” and “or”, operands are “A” , “B”, and “C” each with two possible states (i.e. True/False)
    • Three operands yields a possible 2n test combinations, 8 possible test cases

1.3 Verifying Multi-logic Condition Requirements

 1. If possible, rewrite software requirement to eliminate multi-logic conditions


 2. The preferred method of verification should be test

  • Inspection, Analysis, or Demonstration should only be used if test is not possible


 3. Run a test case for each possible combination of operand values (2n)

  • Test all possible permutations (i.e. cover the full truth table)
  • Note this method is analogous to the Multiple Condition Coverage (MCC) test method used for code unit testing


 4. If running all 2n test cases is not possible (due to hardware limitations or resource constraints):

  • At a minimum, run the subset of tests to prove that varying each operand affects the result, independently
    • Must perform an analysis to determine the correct subset of tests
    • Note this method is analogous to the Modified Condition/Decision Coverage (MCDC) test method used for code unit testing

Example on the following tabs illustrate application of this recommendation. 


 1.4 Determining the Minimum Set of Tests Required

Full set of tests

Example of three operand case [If A and (B or C) then {result}]


  • There are 8 possible permutations (23)
  • Running all eight test cases will completely verify this requirement

Minimum set of Required Tests

  • If running all 8 test cases is not practical, then 4 test cases will ensure that varying each operand affects the result independently
    • To find minimum set of required test cases
      • Take operands one at a time and select test cases where varying only that operand, changes the result.

  • Example of three operand case [If A and (B or C) then {result}]
  • A test cases needed:  2 and 6
    • A changes, B and C are unchanged, and result is changed
  • B test cases needed:  5 and 7
    • B changes, A and C are unchanged, and result is changed
  • C test cases needed:  5 and 6
    • C changes, A and B are unchanged, and result is changed
  • Total possible test cases = 8
  • Total test cases needed   = 4  (2, 5, 6, 7) 

Using same methods used to find MCDC minimum set of unit tests: for n logic conditions (operands) n+1 tests will sufficiently verify the requirement.

 2. Recommendations for Requirements When Conditions Are Not Discrete

Simplified Example:

  • “ If mode is PROCESSING and TEMPERATURE is greater than 50.0, the software shall set a WARNING.”

The example above can be restated as: If A and (B>50.0) then set a warning.

  • In this case, operand A can have two possible states but B can have many states
  • Very difficult to test and analyze all combinations


Best practice is to run tests with values just below the limit, at the limit, and just above the limit.

  • Note this method is analogous to Parameter Boundary Testing
  • For each value of A, vary B just below the limit, at limit, and just above the limit

Analyzing and testing all combinations becomes even more difficult if both A and B can have many states
  • Simplified Example: If (A < 25.0) and (B > 50.0) then set a warning
  • Using the same method as the last example, vary both A and B

Could break this requirement into two separate requirements, for example:
  • If (A < 25.0) then set LO_PRESS = TRUE
  • If (B > 50.0) and LO_PRESS then set a warning
  • In this simplified example, the number of test cases is the same (whether there is one requirement or two), but the test cases are easier to understand.


In cases where a specific operand can have a range of values best practice is to use Boundary Parameter Testing and also select values in the middle of the range

Example:

  • “ If (PRESURE > 25) and (PRESSURE < 45), the software shall set PRESSURE_OK.”



3. Summary - Verifying Multi-logic Condition Requirements

  • Recommend running the full set of 2n test cases for Multi-condition requirements
    • For safety critical requirements, must perform the 2n test cases or request a waiver (to NPR 7150.2  083 and NASA-STD-8739.8A 278)
    • Also recommend required input from Subject Matter Experts/Responsible Engineers to ensure tests adequately cover intent of requirement.

  • If running the full set of test is not practical, run the minimum subset of tests to prove that varying each operand affects the result, independently
  • Why run the full set of 2n test cases instead of the n+1 set?
    • Determining Independent outcome is implementation dependent
      • Therefore, we need to decouple requirements testing from implementation
      • Only way that can be assured is by running all permutations
    • Intensive to analyze and prone to error, and should be decoupled anyway
    • No automated tools for determining which cases to remove based on requirements
    • Relatively low number of these type requirements should exist
      • Should not require many additional test cases

  • For requirements where conditions are not discrete, follow best practice and run tests with values just below the limit, at the limit, and just above the limit.
    • Recommend thorough requirements analysis to ensure all permutations are represented by selected input values.

  • For requirements where conditions have a range of values, follow best practice and run tests with values just below the limit, at the limit, just above the limit, and at least one value in the middle of the range.

See SWE-134 - Safety Critical Software Design Requirements for additional guidance. 

4. Resources

4.1 References

Click here to view master references table.


4.2 Tools


Tools to aid in compliance with this SWE, if any, may be found in the Tools Library in the NASA Engineering Network (NEN). 

NASA users find this in the Tools Library in the Software Processes Across NASA (SPAN) site of the Software Engineering Community in NEN. 

The list is informational only and does not represent an “approved tool list”, nor does it represent an endorsement of any particular tool.  The purpose is to provide examples of tools being used across the Agency and to help projects and centers decide what tools to consider.

5. Lessons Learned

Currently there are no Lessons Learned identified for this topic. 





  • No labels

0 Comments

Page Editor: Tim Crumbley
NASA Official: Tim Crumbley
NASA Software Engineering Handbook - A service of the NASA Office of the Chief Engineer
NASA Privacy Statement
______________________________________________________________________________