First order random forests: Learning relational classifiers with complex aggregates

Abstract

In relational learning, predictions for an individual are based not only on its own properties but also on the properties of a set of related individuals. Relational classifiers differ with respect to how they handle these sets: some use properties of the set as a whole (using aggregation), some refer to properties of specific individuals of the set, however, most classifiers do not combine both. This imposes an undesirable bias on these learners. This article describes a learning approach that avoids this bias, using first order random forests. Essentially, an ensemble of decision trees is constructed in which tests are first order logic queries. These queries may contain aggregate functions, the argument of which may again be a first order logic query. The introduction of aggregate functions in first order logic, as well as upgrading the forest's uniform feature sampling procedure to the space of first order logic, generates a number of complications. We address these and propose a solution for them. The resulting first order random forest induction algorithm has been implemented and integrated in the ACE-ilProlog system, and experimentally evaluated on a variety of datasets. The results indicate that first order random forests with complex aggregates are an efficient and effective approach towards learning relational classifiers that involve aggregates over complex selections.

Complex Aggregates over Clusters of Elements

“…Van Assche et al [21] have made the first implementation of combined aggregates and selections in an ILP system. They have extended the refinement operator of the relational decision tree learner Tilde [3] to include so-called complex aggregates: literals of the form F (V, Q, R), where F is an aggregate function (e.g., count), V is an aggregate variable occurring in the aggregate query Q, and R is the result of applying F to the set of all answer substitutions for V that Q results in (we will call this set the result set of Q).…”

“…A complex aggregate can be constructed by iterative refinement of Q, starting with a general query (e.g., the number of atoms of a molecule) and ending with a very specific one (e.g., the number of carbon atoms bound with an aromatic bond type to an atom with charge larger than 0.06). The feature explosion resulting from combining aggregate functions with selection conditions was handled by upgrading Tilde to a random forest [21] and taking advantage of the feature sampling applied at each node of the trees. Charnay et al [6] have recently proposed an alternative solution, by introducing a hill-climbing approach to build complex aggregates incrementally.…”

“…The latter two types of relationships result in single examples being related to a set of objects, and require mechanisms to deal with such sets. While, traditionally, relational learning approaches handled sets by either looking at properties of individual elements or by aggregating over them, a number of researchers [12,14,17,20,21] have looked into the combination of these approaches, resulting in so-called complex aggregates. Such complex aggregates aggregate over a subset of the elements of a set, where the subset is usually defined by imposing conditions on some attribute values.…”

Feature Discovery with Type Extension Trees

“…The formatting of the feature value emphasizes the hierarchical, combinatorial structure of these count of count values (which can also be nested deeper into count of count of count values, etc.). Note that count of count features in our sense are quite different from nested aggregates in the sense of [1]: unlike the latter, our counts of counts do not aggregate a multiset of values into a single number at each level of nesting.…”

“…Perhaps the count of actors with a proportion of ≥ 50% award winning movies among the movies they participated in is the more relevant feature (here evaluating to 0 and 2, respectively, assuming that the movie to be classified is not considered in the count). Nested aggregates like this have been used in conjunction with first-order decision trees [1]. This very simple example illustrates three interesting aspects of relational features.…”