Thresholding algorithms, maxisets and well-concentrated bases

Kerkyacharian, Gérard; Picard, Dominique; Birgé, Lucien; Hall, Peter A.; Lepski, Oleg; Mammen, Enno; Tsybakov, Alexandre B.

doi:10.1007/bf02595738

Cited by 82 publications

(134 citation statements)

References 38 publications

Supporting

Mentioning

123

Contrasting

Order By: Relevance

“…In our previous example the large parameter set was l 2 ( ) and the previleged subsets were the ellipsoids. Adaptive procedures include adaptive spline smoothing (see Wahba, 1990 for a review), unbiased cross-validation as proposed by Rudemo (1982) and further developed by many authors, soft and hard thresholding methods as initiated by Efroimovich and Pinsker (1984) and further developed by Donoho and Johnstone in a series of papers starting with Donoho and Johnstone (1994), (see references in Donoho and Johnstone, 1998) and by Kerkyacharian and Picard (see, for instance, Kerkyacharian and Picard, 2000 and the references therein), Lepskii's method starting with Lepskii (1990Lepskii ( , 1991 and adaptive local polynomial fit as initiated by Katkovnik (1979) (see a detailed account and recent developments in Nemirovski, 2000). For a recent survey of various approaches to adaptation, we refer to Barron et al (1999, Sect.…”

Section: Curve Estimation and Adaptationmentioning

confidence: 99%

Gaussian model selection

Birgé¹,

Massart²

2001

J. Eur. Math. Soc.

Self Cite

363

442

View full text Add to dashboard Cite

Abstract. Our purpose in this paper is to provide a general approach to model selection via penalization for Gaussian regression and to develop our point of view about this subject. The advantage and importance of model selection come from the fact that it provides a suitable approach to many different types of problems, starting from model selection per se (among a family of parametric models, which one is more suitable for the data at hand), which includes for instance variable selection in regression models, to nonparametric estimation, for which it provides a very powerful tool that allows adaptation under quite general circumstances. Our approach to model selection also provides a natural connection between the parametric and nonparametric points of view and copes naturally with the fact that a model is not necessarily true. The method is based on the penalization of a least squares criterion which can be viewed as a generalization of Mallows' C p . A large part of our efforts will be put on choosing properly the list of models and the penalty function for various estimation problems like classical variable selection or adaptive estimation for various types of l p -bodies. Introducing model selection from a nonasymptotic point of viewChoosing a proper parameter set is a difficult task in many estimation problems. A large one systematically leads to a large risk while a small one may result in the same consequence, due to unduly large bias. Both excessively complicated or oversimplified models should be avoided. The dilemna of the choice, between many possible models, of one which is adequate for the situation at hand, depending on both the unknown complexity of the true parameter to be estimated and the known amount of noise or number of observations, is often a nightmare for the statistician. The purpose of this paper is to provide a general methodology, namely model selection via penalization, for solving such problems within a unified Gaussian framework which covers many classical situations involving Gaussian variables. L. Birgé: UMR 7599 "Probabilités et modèles aléatoires

show abstract

Section: Curve Estimation and Adaptationmentioning

confidence: 99%

Gaussian model selection

Birgé¹,

Massart²

2001

J. Eur. Math. Soc.

Self Cite

363

442

View full text Add to dashboard Cite

show abstract

“…Such risk bounds can be found in Donoho, Johnstone, Kerkyacharian and Picard (1996). The more recent improved results of Kerkyacharian and Picard (2000) do not apply to the L 1 -loss.…”

Section: Two Illustrations With Independent Variablesmentioning

confidence: 99%

Model selection via testing: an alternative to (penalized) maximum likelihood estimators

Birgé¹

2006

Annales de l'Institut Henri Poincare (B) Probability and Statis

189

View full text Add to dashboard Cite

This paper is devoted to the description and study of a family of estimators, that we shall call T -estimators (T for tests), for minimax estimation and model selection. Their construction is based on former ideas about deriving estimators from some families of tests due to Le Cam (1973 and1975) and Birgé (1983, 1984a and and about complexity based model selection from Barron and Cover (1991).It is well-known that maximum likelihood estimators or, more generally, minimum contrast estimators do suffer from various weaknesses, and their penalized versions as well. In particular they are not robust and they require restrictive assumptions on both the models and the underlying parameter for the estimators to work correctly. Our method, which derives an estimator from many simultaneous tests between some probability balls in a suitable metric space, tends to solve many of these difficulties. Its robustness properties allow to deal with minimax estimation and model selection in a unified way, since bounding the minimax risk amounts to perform the method with a single, well-chosen, model. This results in simple bounds for the minimax risk solely based on some metric properties of the parameter space. Moreover the method applies to various statistical frameworks (we shall concentrate here on the i.i.d. and the Gaussian sequence settings only) and can handle essentially all types of models, linear or not, parametric and non-parametric, simultaneously. From these viewpoints, it is much more flexible than traditional methods. In particular, we shall be able to derive some adaptation results for density estimation over Besov balls that do not seem to be accessible to classical methods. The counterpart for these nice properties is the very high computational complexity of our construction which makes our estimators look more like theoretical than practical tools. 0 AMS 1991 subject classifications. Primary 62F35; secondary 62G05.

show abstract

“…We refer to [3], [4], [8], [14], [17], [23], and [24] for general results. Maxisets have also been investigated for other statistical models, see [2] and [25].…”

Section: 2mentioning

confidence: 99%

“…Lemma 2.2 of [17] shows that the spaces W q (0 < q < 2) have other characterizations in terms of wavelet coefficients:…”

Section: 2mentioning

confidence: 99%

Maxisets for Model Selection

et al. 2009

View full text Add to dashboard Cite

Abstract. We address the statistical issue of determining the maximal spaces (maxisets) where model selection procedures attain a given rate of convergence. By considering first general dictionaries, then orthonormal bases, we characterize these maxisets in terms of approximation spaces. These results are illustrated by classical choices of wavelet model collections. For each of them, the maxisets are described in terms of functional spaces. We take a special care of the issue of calculability and measure the induced loss of performance in terms of maxisets.

show abstract

Thresholding algorithms, maxisets and well-concentrated bases

Abstract: Approximation methods, Besov spaces, denoising, minimax rate of convergence, non parametric estimation, oracle inequalities, saturation spaces, wavelet thresholding, 62G05, 62G07, 62G20,

Cited by 82 publications

References 38 publications

Gaussian model selection

Gaussian model selection

Model selection via testing: an alternative to (penalized) maximum likelihood estimators

Maxisets for Model Selection

Contact Info

Product

Resources

About