Testing Reactive Systems Using Behavioural Programming, a Model Centric Approach

Abstract

Testing is a significant aspect of software development. As systems become complex and their use becomes critical to the security and the function of society, the need for testing methodologies that ensure reliability and detect faults as early as possible becomes critical. In academia and in industry, different methods are being developed for improving the testing process. The most promising approach is the model-based approach where a model is developed that defines how the system is expected to behave and how it is meant to react. The tests are derived from the model and an analysis of the test results is conducted based on it.In the proposed doctoral research, we will investigate the prospects of using the Behavioural Programming (BP) modeling approach as an enabler for a model-based testing approach that we will develop. We will develop a natural language (textual and/or graphical) for representing the requirements. The model users will create with our language will be fed to algorithms that we will develop. This includes algorithms for the automatic creation of minimal sets of test cases that cover all of the system's requirements, algorithms for analyzing the results of the tests, and other tools that support the testing process.The focus of our methodology will be to find faults caused by the interaction between different requirements in ways that are difficult for the testers to detect. Specifically, we will focus our attention to concurrency issues such as deadlocks and logical race condition. We will use a variety of methods that are made possible by BP, such as non-deterministic execution of scenarios and use of in-code model-checking for building test scenarios and for finding minimal coverage of the test scenarios for the system requirements using Combinatorial Test Design (CTD) methodologies. We will develop a proof-of-concept tool kit which will allow us to demonstrate and evaluate the above mentioned capabilities. We will compare the performance of our tools with the performance of manual testers and of other model-based tools using comparison criteria that we will define and develop.

Combinatorial Sequence Testing Using Behavioral Programming and Generalized Coverage Criteria

“…All of which are focused on achieving the goal of efficiently testing process, software, and system. A possible trend we have begun to explore is the system modeling process concerning the testing resulting from system requirements using BP tools [3]. This modeling is a prerequisite to the approach presented in this article.…”

“…Email addresses: achiya@bgu.ac.il (Achiya Elyasaf), farchi@il.ibm.com (Eitan Farchi), odedm@post.bgu.ac.il (Oded Margalit), geraw@cs.bgu.ac.il ( Gera Weiss), weissye@post.bgu.ac.il (Yeshayahu Weiss) We propose in the paper to address the above challenge by tackling the following issues: (1) how to describe the space that needs to be covered; (2) how to cover this space with a finite, relatively small, number of tests; and (3) how to methodically explore and exploit knowledge from previous tests in order to optimally reduce bug risks.…”

“…Our answer to the first challenge is described using a proof-of-concept tool that we have implemented whose input language is founded on a scenario-based modeling paradigm, called Behavioral Programming (BP) [1,2]. We show how this modeling language allows us to directly and compositionally specify aspects of the behavior we want to test [3]. Specifically, we demonstrate the automatic creation of composite tests using BP's scenario integration mechanism.…”

“…Examples of how this generalization can be used in various domains are outlines in Section 3. Then, in Section 4, we outline contribution number (3) a Bayesian approach to decide when to exploit previous knowledge and when to explore new possibilities in the context of testing. Contribution number (1), the modeling language that we are proposing for testing, is presented in Section 5 together with a brief description of a proof-of-concept tool that we have developed.…”