Crellvm: verified credible compilation for LLVM

Abstract

Production compilers such as GCC and LLVM are large complex software systems, for which achieving a high level of reliability is hard. Although testing is an effective method for finding bugs, it alone cannot guarantee a high level of reliability. To provide a higher level of reliability, many approaches that examine compilers' internal logics have been proposed. However, none of them have been successfully applied to major optimizations of production compilers. This paper presents Crellvm: a verified credible compilation framework for LLVM, which can be used as a systematic way of providing a high level of reliability for major optimizations in LLVM. Specifically, we augment an LLVM optimizer to generate translation results together with their correctness proofs, which can then be checked by a proof checker formally verified in Coq. As case studies, we applied our approach to two major optimizations of LLVM: register promotion mem2reg and global value numbering gvn, having found four new miscompilation bugs (two in each).

Gauntlet: Finding Bugs in Compilers for Programmable Packet Processing

“…We use random testing to induce crash bugs and we combine it with translation validation and symbolic execution based on formal Z3 semantics to detect semantic bugs using the same programs. This is a departure from prior compiler bug finding that relies almost exclusively on methods from either random testing [26,48] or formal methods [23,30,33].…”

Continuously reasoning about programs using differential Bayesian inference

“…Research on hyperproperties [9] and on relational verification [3] relates the behaviors of a single program on multiple inputs or of multiple programs on the same input. Typical problems studied include equivalence checking [28,34,51,54], information flow security [47], and verifying the correctness of code transformations [27]. Various logical formulations, such as Hoare-style partial equivalence [17], and techniques such as differential symbolic execution [52,54] have been explored.…”

High‐coverage metamorphic testing of concurrency support in C compilers

“…Examples include equivalence‐modulo‐inputs testing (EMI) [21–23], where equivalent programs are derived from an original programe by applying code mutations that, thanks to coverage analysis, are known to have no effect for particular inputs; the application of more general semantics‐preserving transformations to create families of equivalent programs [24‐26]; and generation of programs that are equivalent by construction [27]. Again, concurrency‐induced nondeterminism makes the assumption on which these methods depend unsustainable. Formal approaches have been used to verify the correctness of peephole optimizations in LLVM [28] and to certify that particular LLVM optimizations lead to generation of correct code [29], but these approaches only relate to sequential code optimizations. Ševčík et al [30] and Beringer et al [31] have both extended the CompCert verified compiler [32] to handle concurrent C. However, similar proofs about mainstream compilers would be infeasible. They could also be quite fragile, given that the interpretation of the C11 concurrency semantics is still not a completely settled matter [33,34].…”

“…Formal approaches have been used to verify the correctness of peephole optimizations in LLVM [28] and to certify that particular LLVM optimizations lead to generation of correct code [29], but these approaches only relate to sequential code optimizations.…”