Common Barriers to Model-Based Automation

Posted by Ben Simo

If modeling is as simple as the previous blog entry implies, then why isn’t everyone using model-based automated testing?

1. Model-based testing requires a change in thinking.

Most testers have been trained to transform mental models into explicit test scripts – not document behavior in a machine-readable format. However, most testers will find that modeling is actually easier than defining and maintaining explicit test cases for automation.

2. Large models are very difficult to create and maintain.

Small additions to a model can easily trigger exponential growth in the size and complexity. This state explosion usually requires that large models be defined using code instead of tables. The large model problem can be solved through the use of Hierarchical State Machines (HSMs) and state variables.

Most software states have hierarchical relationships in which child states inherit all attributes of the parent states plus have additional attributes that are specific to the child. Hierarchical state machines reduce redundant specification and allow behavior to be modeled in small pieces that can be assembled into the larger system. For example, the following HSM represents the same keyless entry system with less than half as many transitions defined. Actions that are possible from each state are also possible from all the child states. Validation requirements apply to the parent and all child states. This greatly reduces the size and complexity of the model. Large systems can be modeled by merging many small models.






Defining some state information as variables instead of explicitly named states can reduce the state explosion. Sometimes it is easier to define some conditions as state variables instead of specific child states. These state variables can be used to define guarded transitions. Guarded transitions are transactions that are only possible when the specified data condition is met. A requirement that all doors be closed before the example keyless entry system will arm the alarm may be specified as shown below. Without using guarded transitions, adding the difference in behavior based on whether doors are open or closed would require many new states and transitions.






3. The leading test tool vendors do not offer model-based testing tools.

Modeling is not a “best practice” promoted by the tool vendors. Tool vendors often dictate the way that their tools are used. This results in automation practices being defined to fit the tools instead of making the tools fit the desired approach. The good news is that many test automation tools – both commercial and open source – provide enough flexibility to build new frameworks on top of the built-in functionality.

4. Model-based testing looks complicated.

The model-based testing literature often makes modeling look more complicated than necessary. The truth is that modeling does not require expert mathematicians and computer scientists. A relatively simple framework can support complex test generation and execution with less manual work than most other automation methodologies.