Boosting to correct inductive bias in text classification

Abstract

This paper studies the effects of boosting in the context of different classification methods for text categorization, including Decision Trees, Naive Bayes, Support Vector Machines (SVMs) and a Rocchio-style classifier. We identify the inductive biases of each classifier and explore how boosting, as an error-driven resampling mechanism, reacts to those biases. Our experiments on the Reuters-21578 benchmark show that boosting is not effective in improving the performance of the base classifiers on common categories. However, the effect of boosting for rare categories varies across classifiers: for SVMs and Decision Trees, we achieved a 13-17% performance improvement in macro-averaged F1 measure, but did not obtain substantial improvement for the other two classifiers. This interesting finding of boosting on rare categories has not been reported before.

DERMA: A Melanoma Diagnosis Platform Based on Collaborative Multilabel Analog Reasoning

“…DML is a technique used to identify a suitable distance metric based on the data projection that can be divided into four families [30]. The first two families are based on the supervision of the method: supervised and unsupervised DML.…”

Supporting case-based design for packaged software implementations

“…Possible learning methods include regression models, nearest neighbor classifiers, decision trees, Bayesian probabilistic classifiers, inductive rule learning algorithms, neural networks, online learning approaches, support vector machines, genetic programming techniques, and many hybrid methods. Instead of fixating on a single classification technique, the research will explore ensemble approaches that combine different techniques, such as bagging [2], boosting [5,21] and staged approaches.…”

A new algorithm based on centroid for text categorization

“…Centroid-based algorithm [1] is a commonly used method for text categorization due to the simplicity and linearity. But it often suffers from the inductive bias [2] or model misfit [3] and researches have proposed some methods to further adjust the centroids to make the centroidbased algorithm perform better.…”