Optimal aggregation of classifiers in statistical learning

Abstract

Classification can be considered as nonparametric estimation of sets, where the risk is defined by means of a specific distance between sets associated with misclassification error. It is shown that the rates of convergence of classifiers depend on two parameters: the complexity of the class of candidate sets and the margin parameter. The dependence is explicitly given, indicating that optimal fast rates approaching O(n −1 ) can be attained, where n is the sample size, and that the proposed classifiers have the property of robustness to the margin. The main result of the paper concerns optimal aggregation of classifiers: we suggest a classifier that automatically adapts both to the complexity and to the margin, and attains the optimal fast rates, up to a logarithmic factor.

Model selection by bootstrap penalization for classification

“…Lugosi (2002) and Devroye and Lugosi (1995) then confirmed these reservations by studying the interpolation case where the best classification error inf t∈S P[t(X ) = Y ] of a given class S is nonzero but small. By further analyzing the problem, Mammen and Tsybakov (1999), Tsybakov (2004) and Massart and Nédélec (2005) show that the behavior of the regression function η : x → P[Y = 1|X = x] around 1/2 is crucial. They indeed introduce some margin conditions that can be written in the following general way:…”

Suboptimality of Penalized Empirical Risk Minimization in Classification

“…First, we recall the definition of the margin assumption introduced in [30]. Margin Assumption(MA): The probability measure π satisfies the margin assumption MA(κ), where κ ≥ 1 if we have…”

PAC-Learning with General Class Noise Models

“…The assumption that the noise rates are not equal to 1/2 can be relaxed (at the cost of error values no longer approaching zero) if we assume the weight of the area with noise rate close to 1/2 is bounded (e.g., by applying Tsybakov's noise condition [15]). …”