This paper introduces regular extrapolation, a technique that provides descriptions of systems or system aspects a posteriori in a largely automatic way. The descriptions come in the form of models which offer the possibility of mechanically producing system tests, grading test suites and monitoring running systems. Regular extrapolation builds models from observations via techniques from machine learning and finite automata theory. Also expert knowledge about the system enters the model construction in a systematic way. The power of this approach is illustrated in the context of a test environment for telecommunication systems.

Automatically generated models may provide the key towards controlling the evolution of complex systems, form the basis for test generation and may be applied as monitors for running applications. However, the practicality of automata learning is currently largely preempted by its extremely high complexity and unrealistic frame conditions. By optimizing a standard learning method according to domainspecific structural properties, we are able to generate abstract models for complex reactive systems. The experiments conducted using an industrylevel test environment on a recent version of a telephone switch illustrate the drastic effect of our optimizations on the learning efficiency. From a conceptual point of view, the developments can be seen as an instance of optimizing general learning procedures by capitalizing on specific application profiles.

No abstract

In this paper we present a new coarse grain approach to automated integrated (functional) testing, which combines three paradigms: library-based test design, meaning construction of test graphs by combination of test case components on a coarse granular level, incremental formalization, through successive enrichment of a special-purpose environment for application-specific test development and execution, and library-based consistency checking, allowing continuous verification of application-and aspect-specific properties by means of model checking. These features and their impact for the test process and the test engineers are illustrated along an industrial application: an automated integrated testing environment for CTI-Systems.

No abstract

In this demonstration we illustrate how a posteriori modeling of complex, heterogeneous, and distributed systems is practically performed within an automated integrated testing environment (ITE) to give improved support to the testing process of steadily evolving systems. The conceptual background of the modeling technique, called moderated regular extrapolation is described in a companion paper [3].

No abstract

We demonstrate the Integrated Testing Environment, ITE, an environment for automated and integrated testing at system level. A companion paper [4] describes the problem of systems integrated testing in its conceptual application modelling focus. Here and in [5], the concept's implementation is exemplified in case of automated integrated testing of a Computer Telephony Integration (CTI) system. The CTI system consists of a switch, an automatic call distributor, and a suite of applications forming a call center and supporting the tasks of a center's human agent, e.g., applications that enable an agent to log-on/log-off at the call distributor, or to initiate conference calls with other agents.In the ITE, each hardware and software component of the CTI system is controlled by its own test tool, e.g., a proprietary hardware tracer for the switch or a GUI test tool such as Rational SQA Robot [6] for the applications. Coordinating which action has to be performed by which test tool is under responsibility of the Test Coordinator, a tool built on top of METAFrame Technologies' Agent Building Center (ABC) [2], a general-purpose environment for specification and verification of complex workflows.As outlined in the following sections, the ITE supports the design of test cases including the verification of their consistency, the interactive combination of test cases resulting in test scenarios, and the execution of test cases and test scenarios in the heterogenous processing environment of CTI systems. Design Support FeaturesSystem testing is characterized by focussing on inter-components cooperation. For the design of appropriate system-level test cases it is necessary to know what features the system provides, how to operate the system in order to stimulate a feature, and how to determine if features work. This information is gathered