Overview and Evaluation of Premise Selection Techniques for Large Theory Mathematics

Kühlwein, Daniel; Laarhoven, Twan van; Tsivtsivadze, Evgeni; Urban, Josef; Heskes, Tom

doi:10.1007/978-3-642-31365-3_30

Cited by 44 publications

(76 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been shown that proper design and choice of knowledge selection heuristics can change the overall success of large-theory ATP techniques by tens of percents [1]. This paper continues our work on machine learning algorithms for premise selection [1,5]. We investigate how the knowledge of different proofs can be integrated in the machine learning algorithms for premise selection, and how it influences the performance of the ATPs.…”

Section: Introductionmentioning

confidence: 87%

“…We start with the setting introduced in [1,5]. Γ denotes the set of all first order formulas (usually axioms, definitions and theorems) that appear in a given (fixed) large mathematical corpus (MPTP2078 in this paper).…”

Section: The Machine Learning Framework and The Datamentioning

confidence: 99%

“…This proof matrix, together with the formula features, is used for training machine learning algorithms in [1,5], using MML proofs as ∆. [1] and [5] also contain more details on using machine learning for premise selection.…”

Section: The Machine Learning Framework and The Datamentioning

confidence: 99%

“…In [5] we compared several different premise selection algorithms on the MPTP2078 dataset. Thanks to this comparison we now have ATP proofs for 1328 of the 2078 problems, found by Vampire 0.6 [7].…”

Section: The Machine Learning Framework and The Datamentioning

confidence: 99%

“…In our earlier experiments [5] we tested and evaluated several premise selection algorithms on a subset of the Mizar Mathematical Library (MML), the MPTP2078 large-theory benchmark, 1 using the (human) proofs from the MML as training data for the learning algorithms. We have found that (i) learning from such human proofs helps a lot, but (ii) alternative proofs can quite often be successfully constructed by ATPs, making heuristic methods like SInE surprisingly strong and orthogonal to learning methods.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Learning from Multiple Proofs: First Experiments

Kühlwein

Urban

EPiC Series in Computing

Self Cite

View full text Add to dashboard Cite

Mathematical textbooks typically present only one proof for most of the theorems. However, there are infinitely many proofs for each theorem in first-order logic, and mathematicians are often aware of (and even invent new) important alternative proofs and use such knowledge for (lateral) thinking about new problems.In this paper we start exploring how the explicit knowledge of multiple (human and ATP) proofs of the same theorem can be used in learning-based premise selection algorithms in large-theory mathematics. Several methods and their combinations are defined, and their effect on the ATP performance is evaluated on the MPTP2078 large-theory benchmark. Our first findings are that the proofs used for learning significantly influence the number of problems solved, and that the quality of the proofs is more important than the quantity.

show abstract

Section: Introductionmentioning

confidence: 87%

Section: The Machine Learning Framework and The Datamentioning

confidence: 99%

Section: The Machine Learning Framework and The Datamentioning

confidence: 99%

Section: The Machine Learning Framework and The Datamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Learning from Multiple Proofs: First Experiments

Kühlwein

Urban

EPiC Series in Computing

Self Cite

View full text Add to dashboard Cite

show abstract

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

Chvalovský

Jakubův

Suda

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We describe an efficient implementation of clause guidance in saturation-based automated theorem provers extending the ENIGMA approach. Unlike in the first ENIGMA implementation where fast linear classifier is trained and used together with manually engineered features, we have started to experiment with more sophisticated state-of-the-art machine learning methods such as gradient boosted trees and recursive neural networks. In particular the latter approach poses challenges in terms of efficiency of clause evaluation, however, we show that deep integration of the neural evaluation with the ATP data-structures can largely amortize this cost and lead to competitive real-time results. Both methods are evaluated on a large dataset of theorem proving problems and compared with the previous approaches. The resulting methods improve on the manually designed clause guidance, providing the first practically convincing application of gradient-boosted and neural clause guidance in saturation-style automated theorem provers. IntroductionAutomated theorem provers (ATPs) [32] have been developed for decades by manually designing proof calculi and search heuristics. Their power has been growing and they are already very useful, e.g., as parts of large interactive theorem proving (ITP) verification toolchains (hammers) [5]. On the other hand, with small exceptions, ATPs are still significantly weaker than trained mathematicians in finding proofs in most research domains.Recently, machine learning over large formal corpora created from ITP libraries [37,28,19] has started to be used to develop guidance of ATP systems [39,25,2]. This has already produced strong systems for selecting relevant facts for proving new conjectures over large formal libraries [1,4,9]. More recently, machine learning has also started to be used to guide the internal search of the ATP systems. In sophisticated saturation-style provers this has been done by feedback loops for strategy invention [38,16,33] and by using supervised learning [14,26] to select the next given clause [27]. In the simpler connection tableau systems such as leancop [29], supervised learning has been used to choose ⋆ Supported by the ERC Consolidator grant no. 649043 AI4REASON, and by the Czech project AI&Reasoning CZ.02.1.01/0.0/0.0/15 003/0000466 and the European Regional Development Fund.

show abstract

Machine-Learned Premise Selection for Lean

Piotrowski,

Mir,

Ayers

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We introduce a machine-learning-based tool for the Lean proof assistant that suggests relevant premises for theorems being proved by a user. The design principles for the tool are (1) tight integration with the proof assistant, (2) ease of use and installation, (3) a lightweight and fast approach. For this purpose, we designed a custom version of the random forest model, trained in an online fashion. It is implemented directly in Lean, which was possible thanks to the rich and efficient metaprogramming features of Lean 4. The random forest is trained on data extracted from – Lean’s mathematics library. We experiment with various options for producing training features and labels. The advice from a trained model is accessible to the user via the "Image missing" tactic which can be called in an editor while constructing a proof interactively.

show abstract

Overview and Evaluation of Premise Selection Techniques for Large Theory Mathematics

Cited by 44 publications

References 23 publications

Learning from Multiple Proofs: First Experiments

Learning from Multiple Proofs: First Experiments

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

Machine-Learned Premise Selection for Lean

Contact Info

Product

Resources

About