Model selection by bootstrap penalization for classification

Fromont, Magalie

doi:10.1007/s10994-006-7679-y

Cited by 18 publications

(25 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, the global Rademacher complexities were introduced in order to obtain theoretically validated model selection procedures in classification [45,17]. They are resampling estimates of pen id,g (m) := sup t∈Sm { (P − P n )γ(t) } ≥ (P − P n )γ ( s m (P n ) ) = pen id (m), with Rad weights; more recently, Fromont [34] generalized global Rademacher complexities to a wide family of exchangeable resampling weights and obtained non-asymptotic oracle inequalities. Nevertheless, global complexities (that is, estimates of pen id,g ) are too large compared to pen id so that they cannot achieve fast rates of estimation when the margin condition [53] holds.…”

Section: Related Penalties For Classificationmentioning

confidence: 99%

See 1 more Smart Citation

Model selection by resampling penalization

Arlot¹

2009

Electron. J. Statist.

View full text Add to dashboard Cite

In this paper, a new family of resampling-based penalization procedures for model selection is defined in a general framework. It generalizes several methods, including Efron's bootstrap penalization and the leave-one-out penalization recently proposed by Arlot (2008), to any exchangeable weighted bootstrap resampling scheme. In the heteroscedastic regression framework, assuming the models to have a particular structure, these resampling penalties are proved to satisfy a non-asymptotic oracle inequality with leading constant close to 1. In particular, they are asympotically optimal. Resampling penalties are used for defining an estimator adapting simultaneously to the smoothness of the regression function and to the heteroscedasticity of the noise. This is remarkable because resampling penalties are general-purpose devices, which have not been built specifically to handle heteroscedastic data. Hence, resampling penalties naturally adapt to heteroscedasticity. A simulation study shows that resampling penalties improve on V-fold cross-validation in terms of final prediction error, in particular when the signal-to-noise ratio is not large.Comment: extended version of hal-00125455, with a technical appendi

show abstract

Section: Related Penalties For Classificationmentioning

confidence: 99%

“…When σ min = 0 in (An), Lemma 8 below proves that (A Q ) also holds with D 0 = L (Bg) . Therefore, using (33), (A m,ℓ ) holds with the same D 0 .…”

Section: No Uniform Lower Bound On the Noise-levelmentioning

confidence: 99%

Model selection by resampling penalization

Arlot¹

2009

Electron. J. Statist.

View full text Add to dashboard Cite

show abstract

“…Models can also be selected by means of regularization methods, that is, they are penalizing depending on the number of parameters (Alpaydin, 2004) (Fromont, 2007). Generally, Bayesian learning techniques make use of knowledge on the prior probability distributions in order to assign lower probabilities to models that are more complicated.…”

Section: Introductionmentioning

confidence: 99%

Total Variation Applications in Computer Vision

Estrela

Magalhães

Saotome

2016

Handbook of Research on Emerging Perspectives in Intelligent Pattern Recognition, Analysis, and Image Processing

View full text Add to dashboard Cite

The objectives of this chapter are: (i) to introduce a concise overview of regularization; (ii) to define and to explain the role of a particular type of regularization called total variation norm (TV-norm) in computer vision tasks; (iii) to set up a brief discussion on the mathematical background of TV methods; and (iv) to establish a relationship between models and a few existing methods to solve problems cast as TV-norm. For the most part, image-processing algorithms blur the edges of the estimated images, however TV regularization preserves the edges with no prior information on the observed and the original images. The regularization scalar parameter λ controls the amount of regularization allowed and it is an essential to obtain a high-quality regularized output. A wide-ranging review of several ways to put into practice TV regularization as well as its advantages and limitations are discussed.

show abstract

“…Method 1 above is closely related to the Rademacher complexity approach in learning theory, and our results in this direction are heavily inspired by the work of Fromont [Fro04], who studies general resampling schemes in a learning theoretical setting. It may also be seen as a generalization of crossvalidation methods.…”

Section: The 1 − α Quantile Of the Distribution Of φ Y [ W −W ] Condimentioning

confidence: 99%

“…2. When Y is not assumed to be symmetric and W = 1 a.s., Proposition 2 in [Fro04] shows that (9) holds with E(W 1 − W ) + instead of A W . Therefore, the symmetry of the sample allows us to get a tighter result (for instance twice sharper with Efron or Random hold-out (q) weights).…”

Section: Comparison In Expectationmentioning

confidence: 99%

Resampling-Based Confidence Regions and Multiple Tests for a Correlated Random Vector

Arlot

Blanchard²,

Roquain³

Learning Theory

View full text Add to dashboard Cite

We derive non-asymptotic confidence regions for the mean of a random vector whose coordinates have an unknown dependence structure. The random vector is supposed to be either Gaussian or to have a symmetric bounded distribution, and we observe n i.i.d copies of it. The confidence regions are built using a data-dependent threshold based on a weighted bootstrap procedure. We consider two approaches, the first based on a concentration approach and the second on a direct boostrapped quantile approach. The first one allows to deal with a very large class of resampling weights while our results for the second are restricted to Rademacher weights. However, the second method seems more accurate in practice. Our results are motivated by multiple testing problems, and we show on simulations that our procedures are better than the Bonferroni procedure (union bound) as soon as the observed vector has sufficiently correlated coordinates.

show abstract

Model selection by bootstrap penalization for classification

Cited by 18 publications

References 38 publications

Model selection by resampling penalization

Model selection by resampling penalization

Total Variation Applications in Computer Vision

Resampling-Based Confidence Regions and Multiple Tests for a Correlated Random Vector

Contact Info

Product

Resources

About