Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited

Abstract

Since the late 1990s predicate invention has been under-explored within inductive logic programming due to difficulties in formulating efficient search mechanisms. However, a recent paper demonstrated that both predicate invention and the learning of recursion can be efficiently implemented for regular and context-free grammars, by way of metalogical substitutions with respect to a modified Prolog meta-interpreter which acts as the learning engine. New predicate symbols are introduced as constants representing existentially quantified higher-order variables. The approach demonstrates that predicate invention can be treated as a form of higher-order logical reasoning. In this paper we generalise the approach of meta-interpretive learning (MIL) to that of learning higher-order dyadic datalog programs. We show that with an infinite signature the higher-order dyadic datalog class H 2 2 has universal Turing expressivity though H 2 2 is decidable given a finite signature. Additionally we show that Knuth-Bendix ordering of the hypothesis space together with logarithmic clause bounding allows our MIL implementation Metagol D to PAC-learn minimal cardinality H 2 2 definitions. This result is consistent with our experiments which indicate that Metagol D efficiently learns compact H 2 2 definitions involving predicate invention for learning robotic strategies, the East-West train challenge and NELL. Additionally higher-order concepts were learned in the NELL language learning domain. The Metagol code and datasets described in this paper have been made publicly available on a website to allow reproduction of results in this paper.

Logical Minimisation of Meta-Rules Within Meta-Interpretive Learning

“…randomly chosen subsets of the maximum set of meta-rules. We define Metagol min and Metagol max as variants of Metagol d [10] which use the minimum and maximum sets of meta-rules respectively,…”

“…In [10] specificational meta-rules were referred to simply as meta-rules. In this paper we distinguish between specificational and derivational meta-rules in order to clarify different forms of logical reasoning used in reduction of logical formulae involving first-order and higher-order statements.…”

“…The approach was extended to a fragment of dyadic datalog in [10] and shown to be applicable to problems involving learning kinship relations, robot strategies and dyadic concepts in the NELL database [4]. More recently MIL was adapted to bias reformulation when learning a hierarchy of dyadic string transformation functions for spreadsheet applications [8].…”

How Does Predicate Invention Affect Human Comprehensibility?

“…Failure to address these issues has, until recently, led to limited progress being made in this important topic and many well-known ILP systems such as ALEPH [16] and FOIL [14] have no predicate invention. In the recently introduced Meta-Interpretive Learning (MIL) framework [11,12], predicate invention is conducted via construction of substitutions for meta-rules applied by a meta-interpreter. The use of the meta-rules clarifies the declarative bias being employed.…”

“…Given renewed interest within Inductive Logic Programming (ILP) in the use of predicate invention [11,12,7,1,2] this paper explores the effects on comprehensibility of using anonymous definitions within logic programs. Within experiments we assess students' understanding of such programs within the kinship domain.…”