Theory of Classification: a Survey of Some Recent Advances

Boucheron, Stéphane; Bousquet, Olivier; Lugosi, Gábor

doi:10.1051/ps:2005018

Cited by 354 publications

(416 citation statements)

References 209 publications

Supporting

Mentioning

414

Contrasting

Order By: Relevance

“…Following the ideas initially introduced by Koltchinskii and Panchenko (1999), Bartlett et al (2005) and Bartlett et al (2004) propose some localized versions of Rademacher averages as tight data-dependent measures of complexity. Recently, it has been proved that these localized Rademacher averages can be used to construct margin-adaptive model selection procedures (see Boucheron et al, 2005, for a brief survey, or Koltchinskii, 2003, for a more complete study). In the same spirit, we could introduce localized versions of our bootstrap penalties.…”

Section: Resultsmentioning

confidence: 99%

Model selection by bootstrap penalization for classification

Fromont

2006

Mach Learn

View full text Add to dashboard Cite

We consider the binary classification problem. Given an i.i.d. sample drawn from the distribution of an X × {0, 1}−valued random pair, we propose to estimate the so-called Bayes classifier by minimizing the sum of the empirical classification error and a penalty term based on Efron's or i.i.d. weighted bootstrap samples of the data. We obtain exponential inequalities for such bootstrap type penalties, which allow us to derive non-asymptotic properties for the corresponding estimators. In particular, we prove that these estimators achieve the global minimax risk over sets of functions built from Vapnik-Chervonenkis classes. The obtained results generalize Koltchinskii (2001) and Bartlett et al.'s (2002) ones for Rademacher penalties that can thus be seen as special examples of bootstrap type penalties. To illustrate this, we carry out an experimental study in which we compare the different methods for an intervals model selection problem.

show abstract

Section: Resultsmentioning

confidence: 99%

Model selection by bootstrap penalization for classification

Fromont

2006

Mach Learn

View full text Add to dashboard Cite

show abstract

“…Estimation of a regression function with model selection estimators is considered by Baraud in [5,6], while inverse problems are tackled by Loubes and Ludeña [18,19]. Finally model selection techniques provide nowadays valuable tools in statistical learning (see Boucheron et al [12]). In nonparametric estimation, performances of estimators are usually measured by using the quadratic norm, which gives rise to the following empirical quadratic contrast γ n (u) = −2Y n (u) + u 2 for any function u, where · denotes the norm associated to L 2 (D).…”

Section: Model Selection Proceduresmentioning

confidence: 99%

Maxisets for Model Selection

et al. 2009

View full text Add to dashboard Cite

Abstract. We address the statistical issue of determining the maximal spaces (maxisets) where model selection procedures attain a given rate of convergence. By considering first general dictionaries, then orthonormal bases, we characterize these maxisets in terms of approximation spaces. These results are illustrated by classical choices of wavelet model collections. For each of them, the maxisets are described in terms of functional spaces. We take a special care of the issue of calculability and measure the induced loss of performance in terms of maxisets.

show abstract

“…Step 3-i Empirical probabilities are determined (11). As a result of the functioning of this procedure, any object can be defined in one of two classes with a certain probability, which reflects an uncertainty within data and models of decision rule.…”

Section: Testingmentioning

confidence: 99%

“…A huge amount of work is devoted to this issue. Relevant references to them can be found in monographs [1][2][3][4][5][6][7][8], lectures [9,10] and reviews [11][12][13]. The recent fundamental works [6,14,15] clarify the vast diversity of classification algorithms and its learning procedures.…”

Section: Introductionmentioning

confidence: 99%

Entropy “2”-Soft Classification of Objects

Popkov

Volkovich

Dubnov

et al. 2017

Entropy

View full text Add to dashboard Cite

Abstract:A proposal for a new method of classification of objects of various nature, named "2"-soft classification, which allows for referring objects to one of two types with optimal entropy probability for available collection of learning data with consideration of additive errors therein. A decision rule of randomized parameters and probability density function (PDF) is formed, which is determined by the solution of the problem of the functional entropy linear programming. A procedure for "2"-soft classification is developed, consisting of the computer simulation of the randomized decision rule with optimal entropy PDF parameters. Examples are provided.

show abstract

Theory of Classification: a Survey of Some Recent Advances

Abstract: Abstract. The last few years have witnessed important new developments in the theory and practice of pattern classification. We intend to survey some of the main new ideas that have led to these recent results.Mathematics Subject Classification. 62G08, 60E15, 68Q32.

Cited by 354 publications

References 209 publications

Model selection by bootstrap penalization for classification

Model selection by bootstrap penalization for classification

Maxisets for Model Selection

Entropy “2”-Soft Classification of Objects

Contact Info

Product

Resources

About