Using ensembles for problems with characterizable changes in data distribution: A case study on quantification

Abstract

Ensemble methods are widely applied to supervised learning tasks. Based on a simple strategy they often achieve good performance, especially when the single models comprising the ensemble are diverse. Diversity can be introduced into the ensemble by creating different training samples for each model. In that case, each model is trained with a data distribution that may be different from the original training set distribution. Following that idea, this paper analyzes the hypothesis that ensembles can be especially appropriate in problems that: (i) suffer from distribution changes, (ii) it is possible to characterize those changes beforehand. The idea consists in generate different training samples based on the expected distribution changes, and to train one model with each of them. As a case study, we shall focus on binary quantification problems, introducing ensembles versions for two well-know quantification algorithms. Experimental results show that these ensemble adaptations outperform the original counterpart algorithms, even when trivial aggregation rules are used.

Automatic plankton quantification using deep features

“…In fact, many experiments (Barranquero et al, 2013(Barranquero et al, , 2015González et al, 2017c) have shown that off-the-shelf classifiers are often suboptimal when applied directly to quantification tasks. For that reason, several quantification algorithms have been proposed during the past few years (Firat, 2016;Narasimhan et al, 2016;Pérez-Gállego et al, 2017. A review of quantification learning can be found in González et al (2017b).…”

Maximal fusion of facts on the web with credibility guarantee

“…Active learning approaches allows the learning algorithm to choose the data it wants to get labels, which is suitable in settings where the 650 labels are costly to obtain. It has been applied in various machine learning applications such as information fusion [45,46], speech recognition, information extraction [47] and text classification [48]. In credibility assessment, active learning approaches can be classified into two categories: classifier-independent and classifier-specific.…”

A Review on Quantification Learning

“…The AC correction (32) is sometimes unstable, despite it theoretically returns perfect estimations. When the estimates of tpr and f pr are inaccurate, the correction will cause inappropriate changes [Forman 2008;Pérez-Gállego et al 2017] (this is in part the motivation of the methods in Section 6.5). Using ensembles reduces both risks: the risk of having a bad single classifier and the risk of a poor posterior correction.…”