RBoost: Label Noise-Robust Boosting Algorithm Based on a Nonconvex Loss Function and the Numerically Stable Base Learners

Abstract

AdaBoost has attracted much attention in the machine learning community because of its excellent performance in combining weak classifiers into strong classifiers. However, AdaBoost tends to overfit to the noisy data in many applications. Accordingly, improving the antinoise ability of AdaBoost plays an important role in many applications. The sensitiveness to the noisy data of AdaBoost stems from the exponential loss function, which puts unrestricted penalties to the misclassified samples with very large margins. In this paper, we propose two boosting algorithms, referred to as RBoost1 and RBoost2, which are more robust to the noisy data compared with AdaBoost. RBoost1 and RBoost2 optimize a nonconvex loss function of the classification margin. Because the penalties to the misclassified samples are restricted to an amount less than one, RBoost1 and RBoost2 do not overfocus on the samples that are always misclassified by the previous base learners. Besides the loss function, at each boosting iteration, RBoost1 and RBoost2 use numerically stable ways to compute the base learners. These two improvements contribute to the robustness of the proposed algorithms to the noisy training and testing samples. Experimental results on the synthetic Gaussian data set, the UCI data sets, and a real malware behavior data set illustrate that the proposed RBoost1 and RBoost2 algorithms perform better when the training data sets contain noisy data.

Single image super-resolution via blind blurring estimation and dictionary learning

“…However, fixed number of NNs may cause over-fitting and/or under-fitting phenomenon [7] which is similar to other typical learning issues (e.g. Boosting [8]). …”

“…CDL models the dictionary learning problem as a bilevel optimization problem to minimize the squared loss term on both feature space, namely, one of the optimization problems on the observation feature space and the other on the latent feature space. To balance the reconstruction error on observation feature space and latent feature space, CDL algorithm changes the objective function of [8]:…”

A Correction on Tangentboost Algorithm

“…On the other hand, unbounded increment of penalty value reveals the over…tting problem. Therefore, bounded loss functions and its boosting algorithms have been proposed in the few years [13,14]. TangentBoost is an alternative loss function and the method has bounded loss function.…”

“…As a result, classi…ers can be improper and their generalization ability may not be good. To make classi…ers more stable, some researchers proposed robust boosting algorithms [13,14,16,17,18]. The idea behind TangentBoost algorithm is probability elicitation and conditional risk minimization [19].…”

An algorithm combined with color differential models for license-plate location

“…After training step, a license plate test image located in a natural scene is feed to the system. Then a strong classifier is trained by the adaboost algorithm [7] and is used to classify parts of an image within a search window as either license plate or non-license plate. The original adaboost algorithm is simple and fast; however, it fails to detect license plate when the range of variations for distance or viewing angle increases.…”