A Neurally-Guided, Parallel Theorem Prover

Rawson, Michael; Reger, Giles

doi:10.1007/978-3-030-29007-8_3

Cited by 9 publications

(6 citation statements)

References 20 publications

(1 reference statement)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This will not only speed up inference, but will also make training more efficient as fewer wrong proofs will be considered. This idea is similar to that of neurally-guided theorem proving (Wang et al, 2017;Rawson and Reger, 2019).…”

Section: Parametric Heuristicsmentioning

confidence: 86%

Approximate Inference for Neural Probabilistic Logic Programming

Manhaeve

Marra

Raedt

2021

Proceedings of the Eighteenth International Conference on Principles of Knowledge Representation and Reasoning

View full text Add to dashboard Cite

DeepProbLog is a neural-symbolic framework that integrates probabilistic logic programming and neural networks. It is realized by providing an interface between the probabilistic logic and the neural networks. Inference in probabilistic neural symbolic methods is hard, since it combines logical theorem proving with probabilistic inference and neural network evaluation. In this work, we make the inference more efficient by extending an approximate inference algorithm from the field of statistical-relational AI. Instead of considering all possible proofs for a certain query, the system searches for the best proof. However, training a DeepProbLog model using approximate inference introduces additional challenges, as the best proof is unknown at the start of training which can lead to convergence towards a local optimum. To be able to apply DeepProbLog on larger tasks, we propose: 1) a method for approximate inference using an A*-like search, called DPLA* 2) an exploration strategy for proving in a neural-symbolic setting, and 3) a parametric heuristic to guide the proof search. We empirically evaluate the performance and scalability of the new approach, and also compare the resulting approach to other neural-symbolic systems. The experiments show that DPLA* achieves a speed up of up to 2-3 orders of magnitude in some cases.

show abstract

Section: Parametric Heuristicsmentioning

confidence: 86%

Approximate Inference for Neural Probabilistic Logic Programming

Manhaeve

Marra

Raedt

2021

Proceedings of the Eighteenth International Conference on Principles of Knowledge Representation and Reasoning

View full text Add to dashboard Cite

show abstract

“…As an extension of FormulaNet, [40] construct syntax trees of HOL formulas as structural inputs and use message-passing GNNs to learn features of HOL to guide theorem proving by predicting tactics and tactic arguments at every step of the proof. LERNA [41] uses convolutional neural networks (CNNs) [42] to learn previous proof search attempts (logic formulas) represented by graphs to guide the current proof search for ATP. NeuroSAT [43,44] reads SAT queries (logic formulas) as graphs and learns the features using different graph embedding strategies (e.g.…”

Section: Related Workmentioning

confidence: 99%

Exploring Representation of Horn Clauses using GNNs (technique report)

Liang¹,

Rümmer²,

Brockschmidt³

2022

Preprint

View full text Add to dashboard Cite

In recent years, the application of machine learning in program verification, and the embedding of programs to capture semantic information, has been recognised as an important tool by many research groups. Learning program semantics from raw source code is challenging due to the complexity of real-world programming language syntax and due to the difficulty of reconstructing long-distance relational information implicitly represented in programs using identifiers. Addressing the first point, we consider Constrained Horn Clauses (CHCs) as a standard representation of program verification problems, providing a simple and programming language-independent syntax. For the second challenge, we explore graph representations of CHCs, and propose a new Relational Hypergraph Neural Network (R-HyGNN) architecture to learn program features.We introduce two different graph representations of CHCs. One is called constraint graph (CG), and emphasizes syntactic information of CHCs by translating the symbols and their relations in CHCs as typed nodes and binary edges, respectively, and constructing the constraints as abstract syntax trees. The second one is called control-and data-flow hypergraph (CDHG), and emphasizes semantic information of CHCs by representing the control and data flow through ternary hyperedges.We then propose a new GNN architecture, R-HyGNN, extending Relational Graph Convolutional Networks, to handle hypergraphs. To evaluate the ability of R-HyGNN to extract semantic information from programs, we use R-HyGNNs to train models on the two graph representations, and on five proxy tasks with increasing difficulty, using benchmarks from CHC-COMP 2021 as training data. The most difficult proxy task requires the model to predict the occurrence of clauses in counter-examples, which subsumes satisfiability of CHCs. CDHG achieves 90.59% accuracy in this task. Furthermore, R-HyGNN has perfect predictions on one of the graphs consisting of more than 290 clauses. Overall, our experiments indicate that R-HyGNN can capture intricate program features for guiding verification problems.

show abstract

“…Approaches for first-order logic also differ in tasks they were evaluated on, with some evaluated on offline tasks such as premise selection [47], [59], [62], length prediction [70], and a few in online proof guidance [41], [71], [72], [73]. In online proof guidance, which our work targets, existing work are based on simpler tableaux based reasoners [71], [72], [73]. Unlike these approaches, TRAIL targets guiding efficient, more capable saturation-based theorem provers.…”

Section: Related Workmentioning

confidence: 99%

“…However, approaches that target specific logics such as propositional logic [70], [74] and fragments of first-order logic [75], fail to preserve properties specific to first-order logic. Recent work tried to address this limitation by preserving properties such as invariance to predicate and function argument order [62], [71] and variable quantification [12], [47], [62], [71], [72], [73]. Other approaches specifically target higher-order logics, which have different corresponding graph structures from first-order logic [12], [59].…”

Section: Related Workmentioning

confidence: 99%

Learning to Guide a Saturation-Based Theorem Prover

Abdelaziz¹,

Crouse²,

Makni³

et al. 2021

Preprint

View full text Add to dashboard Cite

Traditional automated theorem provers have relied on manually tuned heuristics to guide how they perform proof search. Recently, however, there has been a surge of interest in the design of learning mechanisms that can be integrated into theorem provers to improve their performance automatically. In this work, we introduce TRAIL, a deep learning-based approach to theorem proving that characterizes core elements of saturation-based theorem proving within a neural framework. TRAIL leverages (a) an effective graph neural network for representing logical formulas, (b) a novel neural representation of the state of a saturation-based theorem prover in terms of processed clauses and available actions, and (c) a novel representation of the inference selection process as an attention-based action policy. We show through a systematic analysis that these components allow TRAIL to significantly outperform previous reinforcement learning-based theorem provers on two standard benchmark datasets (up to 36% more theorems proved). In addition, to the best of our knowledge, TRAIL is the first reinforcement learning-based approach to exceed the performance of a state-of-the-art traditional theorem prover on a standard theorem proving benchmark (solving up to 17% more problems).

show abstract

A Neurally-Guided, Parallel Theorem Prover

Cited by 9 publications

References 20 publications

Approximate Inference for Neural Probabilistic Logic Programming

Approximate Inference for Neural Probabilistic Logic Programming

Exploring Representation of Horn Clauses using GNNs (technique report)

Learning to Guide a Saturation-Based Theorem Prover

Contact Info

Product

Resources

About