Automated regression testing of CTI-systems

Niese, Oliver; Margaria, Tiziana; Hagerer, A.; Steggen, B.; Brune, Georg; Goerigk, Werner; Ide, Hans-Dieter

doi:10.1109/etw.2001.946662

Cited by 13 publications

(5 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The central idea was to exploit our preexisting automated testing machinery [13,18] as a system observation tool for constructing models via (adapted) classical automata learning algorithms [24].…”

Section: Motivationmentioning

confidence: 99%

Knowledge-based relevance filtering for efficient system-level test-based model generation

Margaria

Raffelt²,

Steffen

2005

Innovations Syst Softw Eng

View full text Add to dashboard Cite

Test-based model generation by classical automata learning is very expensive. It requires an impractically large number of queries to the system, each of which must be implemented as a system-level test case. Key in the tractability of observation-based model generation are powerful optimizations exploiting different kinds of expert knowledge in order to drastically reduce the number of required queries, and thus the testing effort. In this paper, we present a thorough experimental analysis of the second-order effects between such optimizations in order to maximize their combined impact. MotivationValidating complex heterogeneous systems escapes established formal approaches, both for system testing and for design verification. Characteristic here in fact is the lack of (formal or semiformal) operational models for many of the hardware and software systems constitute such a scenario. Typical application domains are telecommunication systems and systems on a chip, but also EAI (enterprise application integration) scenarios, which are large software integration projects suffering from an almost complete lack of usable models for the components' behaviors (think, e.g., of an SAP installation with all the customization modules!). This situation is not only due to a lack of care in the production and maintenance of up-to-date models of all the system components. Rather, there is a policy of hiding intellectual prop- T. Margaria (B) erty, since the operational models of the commercial products would reveal too much about protected techniques and designs. Thus the lack of models and the need to "discover" them postproduction will accompany system-level design and testing for a long time to come.Particularly typical in practice are systems that, additionally, include different commercial components that come from various producers and are made available as products, without any model. Our direct industrial experience stems from the area of Computer Telephony Integrated (CTI) systems. In the past we developed a piece of automated testing equipment (ITE) [14,18] that has been used for industrial system-level testing of over 200 COTS applications that interoperate with a family of midrange telecommunication switches. Characteristic here was the absence of any form of (formal or semiformal) operational model for the hardware and software systems constitute a CTI scenario, which are therefore seen and treated as black boxes. In particular, there is no basis for test coverage considerations, focused test suite enhancement, or systematic maintenance support.Fortunately one can observe that the models one needs to "discover" in order to profitably use the components are far smaller than the full model of the implementation: they are much more abstract, since they provide information about only the interface behavior exposed to the environment. In this respect, complexity experts often talk of cognitive complexity (i.e., what a user needs to know in order to use a product), which is much smaller than the technical or intensiona...

show abstract

Section: Motivationmentioning

confidence: 99%

Knowledge-based relevance filtering for efficient system-level test-based model generation

Margaria

Raffelt²,

Steffen

2005

Innovations Syst Softw Eng

View full text Add to dashboard Cite

show abstract

“…Unlike the model checking approach to decide on test consistency [6] we address generic rather than application or domain specific test consistency requirements.…”

Section: Related Work and Conclusionmentioning

confidence: 99%

Test suite consistency verification

Boroday

Petrenko

Ulrich

2008

Proceedings of IEEE East-West Design &Amp; Test Symposium (EWDTS'08)

View full text Add to dashboard Cite

show abstract

“…I.e., in order to be able to resolve the membership queries, abstract sequences of symbols have to be retranslated into concrete stimuli sequences and fed to the system at hand. This problem has been solved with the help of a software enhancement of an existing, industrial test environment [11,12].…”

Section: Learning Finite Automata -Theory and Practicementioning

confidence: 99%

“…Typical examples for this kind of systems are Computer Telephony Integrated (CTI) systems, like complex Call Center solutions, embedded systems, or web-based applications. To actually perform tests, we use a tool called Integrated Test Environment (ITE ) [11,12] which has been applied in research and in industrial practice for different tasks. One important property for regression testing is that results are reproducible, i.e., that results are not subject to accidental changes.…”

Section: Application Domain: Regression Testing Of Complex Reactive mentioning

confidence: 99%

See 1 more Smart Citation

Domain-Specific Optimization in Automata Learning

Hungar

Niese

Steffen

2003

Computer Aided Verification

View full text Add to dashboard Cite

Automatically generated models may provide the key towards controlling the evolution of complex systems, form the basis for test generation and may be applied as monitors for running applications. However, the practicality of automata learning is currently largely preempted by its extremely high complexity and unrealistic frame conditions. By optimizing a standard learning method according to domainspecific structural properties, we are able to generate abstract models for complex reactive systems. The experiments conducted using an industrylevel test environment on a recent version of a telephone switch illustrate the drastic effect of our optimizations on the learning efficiency. From a conceptual point of view, the developments can be seen as an instance of optimizing general learning procedures by capitalizing on specific application profiles.

show abstract

Automated regression testing of CTI-systems

Cited by 13 publications

References 3 publications

Knowledge-based relevance filtering for efficient system-level test-based model generation

Knowledge-based relevance filtering for efficient system-level test-based model generation

Test suite consistency verification

Domain-Specific Optimization in Automata Learning

Contact Info

Product

Resources

About