Generalized test tables: A powerful and intuitive specification language for reactive systems

Abstract

With recent trends in manufacturing automation, such as Industry 4.0, control software in automated production systems becomes more and more complex and volatile, complicating and increasing importance of quality assurance. Test tables are a widely used and generally accepted means to intuitively specify test cases for automation software. However, each table only specifies a single software trace, whereas the actual software behavior may cover multiple similar traces not covered by the table. Within this work, we present a generalization concept for test tables allowing for bounded and unbounded repetition of steps, "don't-care" values, as well as calculations with earlier observed values. We provide a verification mechanism for checking conformance of an IEC 61131-3 PLC software with a generalized test table, making use of a state-of-the-art model checker. Our notation is inspired by widely-used paradigms found in spreadsheet applications. By an empirical study with mechanical engineering students, we show that the notation matches user expectations. A real-world example extracted from an industrial automation plant illustrates our approach.

Table‐based formal specification approaches for control engineers—empirical studies of usability

“…This experiment (Exp-1) consists of a one-hour tutorial of the target language, that is, GTTs, given to the participants in the form of a lecture allowing them to ask questions and including a review of examples. Tasks were given to the participants F I G U R E 1 An example of a Generalised Test Tables (GTT; derived from [16]) separately in the target language (GTT) and the control language (PN) for comparison. The first experiment was also planned as a preliminary measurement of the appropriate level of tasks.…”

“…Ljungkrantz et al [11] even claim that 'today's need for control software developers to understand mechanical and electrical drawings might also be less important if better specifications were available. ' To tackle this problem, we have suggested a formal specification language in the recent works [16,17], namely Generalised Test Tables (GTTs). The origin of GTT lies in test tables that are the industry's practices to describe test cases in the table form [18,19]; and GTT can be used as a formal language for verification purposes supplementing testing by generalising test tables.…”

“…To tackle this problem, we have suggested a formal specification language in the recent works [16, 17], namely Generalised Test Tables (GTTs). The origin of GTT lies in test tables that are the industry's practices to describe test cases in the table form [18, 19]; and GTT can be used as a formal language for verification purposes supplementing testing by generalising test tables.…”

Runtime Verification of Generalized Test Tables

“…The Linear Regression function block implements a commonly needed functionality for implementing CPSs, namely the calibration of sensor values. The evaluated software module origins from [18], where it is used to demonstrate static verification of GTTs using model checking. The state space of this function block, which uses floating point arithmetic, is relatively large and its state transition function relatively complex, limiting the applicability of static verification tools.…”

“…Generalized test table of a system which maps sensor values to their physical representation by a taught linear curve. Originally presented in[18].…”