Supervised and Unsupervised Discretization of Continuous Features

Abstract

Many supervised machine learning algorithms require a discrete feature space. In this paper, we review previous work on continuous feature discretization, identify de ning characteristics of the methods, and conduct an empirical evaluation of several methods. We compare binning, an unsupervised discretization method, to entropy-based and purity-based methods, which are supervised algorithms. We found that the performance of the Naive-Bayes algorithm signi cantly improved when features were discretized using an entropy-based method. In fact, over the 16 tested datasets, the discretized version of Naive-Bayes slightly outperformed C4.5 on average. We also show that in some cases, the performance of the C4.5 induction algorithm signi cantly improved if features were discretized in advance; in our experiments, the performance never signi cantly degraded, an interesting phenomenon considering the fact that C4.5 is capable of locally discretizing features.

Rough Sets and Rough Logic: A KDD Perspective

“…For some tables, taking into account the small number of their objects, we have adopted the approach based on five-fold crossvalidation (CV − 5). The obtained results (Table 3) can be compared with those reported in [21,69] (Table 2). For predicting decisions on new cases we apply only decision rules generated either by the decision tree (using hyperplanes) or by rules generated in parallel with discretization.…”

“…For predicting decisions on new cases we apply only decision rules generated either by the decision tree (using hyperplanes) or by rules generated in parallel with discretization. For some tables the classification quality of our algorithm is better than that of the C4.5 or Naive-Bayes induction algorithms [100] even when used with different discretization methods [21,69,15].…”

Inference in hybrid Bayesian networks

“…Discretization has been widely studied from both a general point of view [26,27] and aimed specifically at BNs [28,29] and classification problems [30,31]. Discretization amounts to replacing a continuous variable X in a model by its discrete counterpart X 0 .…”

Encoding Ordinal Features into Binary Features for Text Classification

“…Our encoding of ordinal features into binary features is reminiscent of machine learning algorithms for discretizing a continuous (i.e., real-valued) feature t k (see [5] for a survey and experimental comparison of less-than-recent methods, and [6,7] for two more recent surveys). These algorithms attempt to optimally subdivide the interval [α k , β k ] on which a feature t k ranges (where the interval [α k , β k ] may or may not be the same for all features in the feature space) into…”