Model Dependent Test Coverage Variabilities

Abstract

During testing as much functionality of the System Under Test (SUT) as possible shall be executed in order to determine its functional correctness. The test coverage on behavioral models is a metric for such a determination, but can be misleading if the wrong model is used for its calculation. For this reason the model dependent test coverage variabilities have to be known. In this paper we investigate the impact of model transformation steps from Unified Modeling Language (UML) state machines to Extended Symbolic Transition Systems (ESTSs) on the test coverage and discuss variabilities if other models like Extended Finite State Machines (EFSMs) are used. We qualitatively compare the syntax of models used for model-based testing and their coverage variabilities. We propose solutions to mitigate them in order to support model selection. The conducted analysis identifies the model syntax elements causing the coverage variabilities, which are experimentally confirmed on an industrial use case using the test case generation tool STATION.