Optimal rates and adaptation in the single-index model using aggregation

Gaı̈ffas, Stéphane; Lecué, Guillaume

doi:10.1214/07-ejs077

Cited by 42 publications

(33 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, convex aggregation methods typically depend on some tuning parameter (such as the temperature in the case of a Gibbs prior and a Bayesian aggregation procedure [4,10]. Of course, one is led to choose the tuning parameter that minimizes the empirical risk, and this choice turns out to be a rather efficient one, as shown by some empirical studies in [7]. Nevertheless, there is no known theoretical result on the choice of the tuning parameter.…”

Section: Question 12 Is Empirical Minimization Performed On Conv(f) mentioning

confidence: 99%

Aggregation via empirical risk minimization

Lecué

Mendelson

2008

Probab. Theory Relat. Fields

Self Cite

View full text Add to dashboard Cite

Given a finite set F of estimators, the problem of aggregation is to construct a new estimator whose risk is as close as possible to the risk of the best estimator in F. It was conjectured that empirical minimization performed in the convex hull of F is an optimal aggregation method, but we show that this conjecture is false. Despite that, we prove that empirical minimization in the convex hull of a well chosen, empirically determined subset of F is an optimal aggregation method.

show abstract

Section: Question 12 Is Empirical Minimization Performed On Conv(f) mentioning

confidence: 99%

Aggregation via empirical risk minimization

Lecué

Mendelson

2008

Probab. Theory Relat. Fields

Self Cite

View full text Add to dashboard Cite

show abstract

“…He derive asymptotic results for projection estimators, in the specic case of projection pursuit constraints. More recently, Gaïas and Lecué (2007) compute minimax rates and build an aggregation procedure with local polynomial estimates, which attains these rates. Lepski's methods are probably rst used by Goldenshluger and Lepski (2009) in the multidimensional white noise model: the results can be applied to several structural models (including the single-index model as well as additive models, or multi-index models), under restricted smoothness assumptions on the function to recover.…”

Section: Concluding Remarks and Outlookmentioning

confidence: 99%

Adaptive estimation in the functional nonparametric regression model

Chagny

Roche

2016

Journal of Multivariate Analysis

View full text Add to dashboard Cite

In this paper, we consider nonparametric regression estimation when the predictor is a functional random variable (typically a curve) and the response is scalar. Starting from a classical collection of kernel estimates, the bias-variance decomposition of a pointwise risk is investigated to understand what can be expected at best from adaptive estimation. We propose a fully data-driven local bandwidth selection rule in the spirit of the Goldenshluger and Lepski method. The main result is a nonasymptotic risk bound which shows the optimality of our tuned estimator from the oracle point of view. Convergence rates are also derived for regression functions belonging to Hölder spaces and under various assumptions on the rate of decay of the small ball probability of the explanatory variable. A simulation study also illustrates the good practical performances of our estimator.

show abstract

“…Precise risk bounds for the EWA in the model of regression with fixed design have been established in (Chernousova et al, 2013;Dai et al, 2014;Dalalyan and Salmon, 2012;Dalalyan and Tsybakov, 2007, 2012bGolubev and Ostrovski, 2014;Leung and Barron, 2006). In the model of regression with random design, the counterpart of the EWA, often referred to as mirror averaging, has been thoroughly studied in (Audibert, 2009;Chesneau and Lecué, 2009;Dalalyan and Tsybakov, 2012a;Gaïffas and Lecué, 2007;Juditsky et al, 2008;Lecué and Mendelson, 2013;Yuditskiȋ et al, 2005). Note that when the temperature τ equals σ 2 /n, the EWA coincides with the Bayesian posterior mean in the regression model with Gaussian noise provided that the prior is defined by π 0 (β) ∝ exp(−λ Pen(β)/τ ).…”

Section: Introductionmentioning

confidence: 99%

On the exponentially weighted aggregate with the Laplace prior

Dalalyan

Grappin²,

Paris

2018

Ann. Statist.

View full text Add to dashboard Cite

In this paper, we study the statistical behaviour of the Exponentially Weighted Aggregate (EWA) in the problem of high-dimensional regression with fixed design. Under the assumption that the underlying regression vector is sparse, it is reasonable to use the Laplace distribution as a prior. The resulting estimator and, specifically, a particular instance of it referred to as the Bayesian lasso, was already used in the statistical literature because of its computational convenience, even though no thorough mathematical analysis of its statistical properties was carried out. The present work fills this gap by establishing sharp oracle inequalities for the EWA with the Laplace prior. These inequalities show that if the temperature parameter is small, the EWA with the Laplace prior satisfies the same type of oracle inequality as the lasso estimator does, as long as the quality of estimation is measured by the prediction loss. Extensions of the proposed methodology to the problem of prediction with low-rank matrices are considered.MSC 2010 subject classifications: Primary 62J05; secondary 62H12.

show abstract

Optimal rates and adaptation in the single-index model using aggregation

Cited by 42 publications

References 24 publications

Aggregation via empirical risk minimization

Aggregation via empirical risk minimization

Adaptive estimation in the functional nonparametric regression model

On the exponentially weighted aggregate with the Laplace prior

Contact Info

Product

Resources

About