Rule-based Machine Learning Methods for Functional Prediction

Abstract

We describe a machine learning method for predicting the value of a real-valued function, given the values of multiple input variables. The method induces solutions from samples in the form of ordered disjunctive normal form DNF decision rules. A central objective of the method and representation is the induction of compact, easily interpretable solutions. This rule-based decision model can be extended to search e ciently for similar cases prior to approximating function values. Experimental results on real-world data demonstrate that the new techniques are competitive with existing machine learning and statistical methods and can sometimes yield superior regression performance.

Generating Rule Sets from Model Trees

“…Decision lists presented in the If-Then rule format are one of the most popular description languages used in machine learning. They have the potential to be more compact and more predictive than their tree counterparts (WI95). In any application, the desired outcome is a small descriptive model which has strong predictive capability.…”

Instance-Based Regression by Partitioning Feature Projections

“…In this section, we summarize its properties in comparison with other important approaches in the literature. The approaches considered in the literature are instancebased regression, KNN [18], locally weighted regression, LOESS [19], rule-based regression, RULE [8], projection pursuit regression, PPR [13], partitioning algorithms that induce decision trees, CART [10], DART [11] and multivariate adaptive regression splines, MARS [12]. Properties of RPFP and the other seven approaches are summarized in Table 1.…”

“…KNN and partitioning approaches such as rule-based regression [8,9], treebased regression [10,11] and MARS [12] are such techniques. Among projection-based methods, only projection pursuit regression, PPR [13], handles interactions with the following model.…”

“…The weight measure (8) implies that each instance is weighted according to the inverse square of its distance from the query. The weighted linear least squares approximation is not appropriate for categorical features.…”

“…Some research on classification methods and real data sets shows that generally the main effects of the features are sufficient to determine the target values [3,8]. If some features have only main effects on targets, RPFP makes predictions for those features by using the whole instance space (instead of the local region determined by partitioning) since a large number of training instances allows better approximations.…”