Generating correctness proofs with neural networks

Abstract

Foundational verification allows programmers to build software which has been empirically shown to have high levels of assurance in a variety of important domains. However, the cost of producing foundationally verified software remains prohibitively high for most projects, as it requires significant manual effort by highly trained experts. In this paper we present Proverbot9001, a proof search system using machine learning techniques to produce proofs of software correctness in interactive theorem provers. We demonstrate Prover-bot9001 on the proof obligations from a large practical proof project, the CompCert verified C compiler, and show that it can effectively automate what were previously manual proofs, automatically producing proofs for 28% of theorem statements in our test dataset, when combined with solver-based tooling. Without any additional solvers, we exhibit a proof completion rate that is a 4X improvement over prior state-of-the-art machine learning models for generating proofs in Coq.

Deductive Synthesis of Programs with Pointers: Techniques, Challenges, Opportunities

“…Future Directions. To guide the search in a more intelligent and flexible way, we turn to extensive recent work on using learned models to guide proof search [8,28,49,78,95] and program synthesis [5,15,39,46,55,82]. Guiding deductive synthesis would most likely require a non-trivial combination of these two lines of work.…”

“…In the area of proof search, existing techniques are used to select the next strategy in a proof assistant script [59,60,78,95], or select a subset of clauses to use in a first-order resolutions proof [9,49]. Although these techniques are not directly applicable to our context, we can likely borrow some high-level insights, such as two-phased search [49], which applies a slow neural heuristic to make important decisions in early stages of search (e.g., which predicate instances to unfold), and then less accurate but much faster hand-coded heuristics take over.…”

The Tactician (extended version): A Seamless, Interactive Tactic Learner and Prover for Coq

“…A probable cause is the slowness of deep neural networks which is common to most proving experiments geared towards the deep learning community. Proverbot9001 is a proof search system for Coq based on a neural architecture [27]. The system is evaluated on the verified CompCert compiler [22].…”

First Experiments with Neural cvc5

“…In recent years, machine learning (ML) and neural methods have been increasingly used to guide the search procedures of automated theorem provers (ATPs). Such methods have been so far developed for choosing inferences in connection tableaux systems [50,27,29,37,51], resolution/superposition-based systems [24,23,20,49], SAT solvers [48], tactical ITPs [17,3,5,18,30,42,40] and most recently also for the iProver [31] instantiation-based system [9]. In SMT (Satisfiability Modulo Theories), ML has so far been mainly used for tasks such as portfolio and strategy optimization [47,36,2].…”