Derivation Reduction of Metarules in Meta-interpretive Learning

Abstract

Meta-interpretive learning (MIL) is a form of inductive logic programming. MIL uses second-order Horn clauses, called metarules, as a form of declarative bias. Metarules define the structures of learnable programs and thus the hypothesis space. Deciding which metarules to use is a trade-off between efficiency and expressivity. The hypothesis space increases given more metarules, so we wish to use fewer metarules, but if we use too few metarules then we lose expressivity. A recent paper used Progol's entailment reduction algorithm to identify irreducible, or minimal, sets of metarules. In some cases, as few as two metarules were shown to be sufficient to entail all hypotheses in an infinite language. Moreover, it was shown that compared to non-minimal sets, learning with minimal sets of metarules improves predictive accuracies and lowers learning times. In this paper, we show that entailment reduction can be too strong and can remove metarules necessary to make a hypothesis more specific. We describe a new reduction technique based on derivations. Specifically, we introduce the derivation reduction problem, the problem of finding a finite subset of a Horn theory from which the whole theory can be derived using SLD-resolution. We describe a derivation reduction algorithm which we use to reduce sets of metarules. We also theoretically study whether certain sets of metarules can be derivationally reduced to minimal finite subsets. Our experiments compare learning with entailment and derivation reduced sets of metarules. In general, using derivation reduced sets of metarules outperforms using entailment reduced sets of metarules, both in terms of predictive accuracies and learning times.

Logical reduction of metarules

“…Fourth, we describe the reduction algorithms and discuss their computational complexity. Finally, we corroborate the experimental results of Cropper and Tourret on Michalski's train problem [16] and provide additional experimental results on two more domains: real-world string transformations and inducing Datalog game rules from observations.…”

“…As mentioned in the introduction, entailment reduction can be too strong a form of reduction. We therefore describe a new form of reduction based on derivability [16,61]. Although our notion of derivation reduction can be defined for any proof system (such as unconstrained resolution as is done in [61]) we focus on SLD-resolution because we want to reduce sets of metarules, which are definite clauses.…”

“…However, in some cases, typically where the fragments contained many metarules, the algorithms took around 12 hours to find a reduction. By contrast, the subsumption reduction algorithm typically found a reduction in 30 minutes 16. Connected clauses are also known as linked clauses[27].…”

Learning higher-order logic programs

“…For instance, consider learning regular grammars, such as a * b * c * . To improve learning efficiency it would be desirable to encode the concept of Kleene star operator 12 as a higher-order definition, such as: kstar(P,A,A). kstar(P,A,B):call(P,A,C), kstar(P,C,B).…”

“…Curry metarules can also be used when learning first-order programs, where existentially quantified argument variables could be bound to constant symbols, rather than predicate symbols. In other words, this issue is a limitation of MIL but manifests itself clearly when learning higher-order programs 12. The Kleene star operator represents zero-or-more repetitions (here applications) of its argument.…”