Using Bounded Model Checking for Coverage Analysis of Safety-Critical Software in an Industrial Setting

Abstract

Testing and Bounded Model Checking (BMC) are two techniques used in Software Verification for bug-hunting. They are expression of two different philosophies: testing is used on the compiled code and it is more suited to find errors in common behaviors, while BMC is used on the source code to find errors in uncommon behaviors of the system. Nowadays, testing is by far the most used technique for software verification in industry: it is easy to use and even when no error is found, it can release a set of tests certifying the (partial) correctness of the compiled system. In the case of safety critical software, in order to increase the confidence of the correctness of the compiled system, it is often required that the provided set of tests covers 100% of the code. This requirement, however, substantially increases the costs associated to the testing phase, since it often involves the manual generation of tests. In this paper we show how BMC can be productively applied to the Software Verification process in industry. In particular, we show how to productively use a Bounded Model Checker for C programs (CBMC) as an automatic test generator for the Coverage Analysis of Safety Critical Software.

Incremental test case generation using bounded model checking: an application to automatic rating

“…In the experiments we used the CBMC tool [5], and as input for the experiments we used the database of exercises and solutions obtained from an industrial tool for automatic rating developed by Codility. 1 All the solutions we used were written in the C language.…”

“…Furthermore, the possibility of application of model checkers for automated program testing is receiving increasing attention in the literature; see, for instance, [1,3,9,[11][12][13][14]. The strongest point of model checking is its capacity for the fully automated generation of test suites (see, for instance, [3]); on the other hand, some drawbacks resulting from the use of model checkers in test case generation are discussed in [9], originating in the fact that software model checking is not in a fully mature state yet.…”

“…As reported in [1], the CBMC tool has been used as a test generator for the coverage analysis (cf. [3]) of software, yielding a substantial reduction in costs of the testing phase in an industrial setting.…”

Formal Verification and Validation of ERTMS Industrial Railway Train Spacing System

“…A comprehensive survey can be found in [20]. Particularly for the verification of ERTMS, the work closely related to ours includes [1,13,24]. The work in [13] covers the whole safety logic of the interlocking application via manual translations into PROMELA, the language of the SPIN model checker.…”

“…The work in [1] shows the use of BMC approaches, via CBMC, to automatically generate test suites for the coverage analysis of safety-critical ERTMS. The problem of generating test suite can be reduced to the problem of verifying multiple assertions.…”