Automatic Testing of LUSTRE/SCADE Programs

Abstract

The design of embedded systems in safety-critical domains is a demanding task. During the last decades, formal methods and model-based approaches are widely used for the design, the analysis, the implementation and testing of major industrial applications. The work in this thesis addresses the improvement of the testing process with a view to automating test data generation as well as its quality evaluation, in the framework of reactive synchronous systems specified in the Lustre/SCADE language. The proposed testing methods use two different requirement-based models of the program under test. On the one hand, we present a testing methodology using the Lutess tool that automatically generates test input data based exclusively on the environment description of the system under test. In particular, this environment description is derived from the system informal requirements and represents a test model of the program describing its functional behavior. Several test models can be built for the same program simulating certain system failures or different situations that must be tested. Test models are then used to guide the test data generation process towards meaningful and valid test input sequences with regard to the specifications. On the other hand, among the large set of coverage criteria that have been proposed in the last years, we study the extension of a coverage assessment technique especially dedicated to Lustre/SCADE applications, using LUSTRUCTU, a prototype tool for coverage measurement. In this case, we are based on the SCADE model of the program under test built from the system requirements. The coverage criteria are defined directly on the Lustre/SCADE specifications and coverage is measured on the SCADE model rather than on the generated C code. The extended criteria take into account two new aspects: the use of multiple clocks in a Lustre program as well as integration testing as opposed to unit testing, allowing the coverage measurement of large-sized systems. These two strategies targeting, respectively, the test data generation process and the coverage evaluation are tailored to industrial needs and could have a positive impact in effectively testing real-world applications. Case studies extracted from the avionics domain are used to demonstrate the application of these methods as well as to empirically evaluate their performance and complexity. "If we knew what it was we were doing, it would not be called research, would it ?"