A Deep Reinforcement Learning Approach to First-Order Logic Theorem Proving

Abstract

Automated theorem provers have traditionally relied on manually tuned heuristics to guide how they perform proof search. Deep reinforcement learning has been proposed as a way to obviate the need for such heuristics, however, its deployment in automated theorem proving remains a challenge. In this paper we introduce TRAIL, a system that applies deep reinforcement learning to saturation-based theorem proving. TRAIL leverages (a) a novel neural representation of the state of a theorem prover and (b) a novel characterization of the inference selection process in terms of an attention-based action policy. We show through systematic analysis that these mechanisms allow TRAIL to significantly outperform previous reinforcement-learning-based theorem provers on two benchmark datasets for first-order logic automated theorem proving (proving around 15% more theorems).

BAIT: Benchmarking (Embedding) Architectures for Interactive Theorem-Proving

“…Their results also include synthetic proof logs to improve performance, while we only trained with human proof logs. The current stateof-the-art for the HOLStep and MIZAR40 benchmarks remains the DAG-LSTM from (Crouse et al 2020). As this architecture cannot compute separate embeddings for goals and premises, which makes prediction computationally intractable for end-to-end systems (Wu et al 2021a;Paliwal et al 2020;Yang et al 2023)…”

“…The embedding model is critical, being used to encode ITP expressions for subsequent tactic, premise and goal selection. Current results either use graph based approaches (Kaliszyk, Chollet, and Szegedy 2017;Paliwal et al 2020;Crouse et al 2020), or treat expressions as a sequence (Lample et al 2022;Polu et al 2023;Han et al 2022), with no thorough comparison between them across ITP systems. INT (Wu et al 2021b) provides the only comparison, in a synthetic proving environment, without directly isolating the embedding architecture.…”

Neuro-Symbolic Techniques for Description Logic Reasoning (Student Abstract)

“…Secondly, automatic learning should be incorporated. Recently, reinforcement learning for FOL [78] has shown tremendous improvements where the systems learn to perform reasoning from scratch. For instance, in the case of reasoners that use inference-rule based methods, instead of providing the inferences (output) beforehand and then mapping the input and the output sequences using seq-2-seq translators, an RL agent can be provided with all the ontology axioms, along with the inference rules.…”

Combining data and theory for derivable scientific discovery with AI-Descartes

“…Moreover, the run-time variance of a theorem prover is very large: the system can at times solve some "large" problems while having difficulties with some "smaller" problems. Recent developments in the neuro-symbolic area use deeplearning techniques to enhance standard theorem provers (e.g., see Crouse et al 8 ). We are still at the early stages of this research and there is still a lot that can be done.…”

“…Moreover, deriving models from a logical theory using formal reasoning tools is especially difficult when arithmetic and calculus operators are involved (e.g., see the work of Grigoryev et al 7 for the case of inequalities). Machine-learning techniques have been used to improve the performance of ATPs, for example, by using reinforcement learning to guide the search process 8 . This research area has received much attention recently [9][10][11] .…”