Implementing Conformiq Qtronic

Abstract

Abstract. Conformiq Qtronic1 is a commercial tool for model driven testing. It derives tests automatically from behavioral system models. These are black-box tests [1] by nature, which means that they depend on the model and the interfaces of the system under test, but not on the internal structure (e.g. source code) of the implementation.In this essay, which accompanies my invited talk, I survey the nature of Conformiq Qtronic, the main implementation challenges that we have encountered and how we have approached them. Problem StatementConformiq Qtronic is an ongoing attempt to provide an industrially applicable solution to the following technical problem: Create an automatic method that provided an object (a model) M that describes the external behavior of an open system (begin interpreted via well-defined semantics) constructs a strategy to test real-life black-box systems (implementations) I in order to find out if they have the same external behavior as M .(Now the fact that we have created a company with the mission to solve this technical problem hints that we believe it to be a good idea to employ automatic methods of this kind in actual software development processes-which is not a priori self-evident. In this essay, however, I will not touch these commercial and methodological issues, but will concentrate on the technical problem only.)Two notes are due, and the first concerns the test harness. It is namely usually so that the model M describes a system with different interfaces and behavior than the real implementation that is being tested (iut, implementation under test). Virtually always, the model M is somehow described on a higher level of abstraction than the real implementation. As a concrete example, sip (Session Initiation Protocol) [2] is a packet-oriented protocol that is run over a transport (e.g. udp [3]) and that has an ascii encoding: every sip packet is encoded as a series of ascii characters. However, some possible model M for testing an sip implementation could define the logical flow of packets with their main contents but not describe, for instance, their actual encoding. How can this model M be used to test a real sip implementation that requires ascii encoded packets over udp? The answer is that the "abstract" tests generated automatically from 1 www.conformiq.com A.

A Comparison of Three Black-Box Optimization Approaches for Model-Based Testing

“…Tools such as Conformiq [3], Uppaal [4], and Spec Explorer [5] use customized modelling languages, and static analysis-based approach with symbolic execution and constraint solving to optimize test sets. Mostly these approaches target offline test generation, while some work [6,4] has also made use of twophased test generation where static analysis is first performed and the resulting information is used to aid in online test generation.…”

“…Many optimization solutions [2,3,4,1,5] use a custom notation combined with a specialized runtime environment to represent the test model, allowing for performing an in-depth white-box static analysis, e.g., symbolic execution, of the model. Tools such as constraint solvers can then be applied on this information to find test cases that yield short paths to reach given coverage targets [3,6,5]. Some use manually crafted scenarios to guide the generator towards specific paths [2].…”

On theorem prover-based testing

“…Moreover, there are commercial test tools that are especially designed for the use in industrial applications, for example, LEIRIOS 6 Test Generator [JL07] or ConformiQ Qtronic [Hui07]. Both tools support a high-level system specification, written in B and UML/OCL (LEIRIOS Test Generator) or annotated state-sequence charts (ConformiQ Qtronic) and apply symbolic constraint solving techniques for generating test cases that cover all symbolic execution paths of the specification.…”

Generating models of infinite-state communication protocols using regular inference with abstraction

“…However, parameters have a significant impact on control flow in typical protocols: they can be sequence numbers, configuration parameters, agent and session identifiers, etc. The influence of data on control flow is taken into account by model-based test generation tools, such as ConformiQ Designer [27] and Spec Explorer [50,17]. It is therefore important to extend inference techniques to handle message alphabets and state-spaces with structures containing data parameters with large domains.…”