Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems

Koltchinskii, Vladimir; Saint-Flour, École de probabilités de d'été

doi:10.1007/978-3-642-22147-7

Cited by 357 publications

(530 citation statements)

References 105 publications

Supporting

Mentioning

522

Contrasting

Order By: Relevance

“…More generally, random models are pervasive in the analysis of statistical estimation proce-dures for high-dimensional data. Random matrix theory plays a key role in this field [123,133,101,31].…”

Section: Stochastic Block Modelmentioning

confidence: 99%

An Introduction to Matrix Concentration Inequalities

Tropp

2015

FNT in Machine Learning

339

159

View full text Add to dashboard Cite

Section: Stochastic Block Modelmentioning

confidence: 99%

An Introduction to Matrix Concentration Inequalities

Tropp

2015

FNT in Machine Learning

339

159

View full text Add to dashboard Cite

“…This is a regularization effect which is due to the convex loss φ. In fact, proof of Theorem 2.2 relies on the powerful model selection machinery presented in Blanchard et al (2008) coupled with modern empirical process theory arguments developed in Koltchinskii (2011). We also emphasize that a concrete but suboptimal value of the constant C may be deduced from the proof, but that no attempt has been made to optimize this constant.…”

Section: Theorem 21 Let ξ Be the Random Variable Defined Bymentioning

confidence: 99%

“…Using the symmetrization inequality presented in Theorem 2.1 of Koltchinskii (2011), it is easy to see that…”

Section: Proof Of Theorem 22mentioning

confidence: 99%

Cox process functional learning

Biau

Cadre

Paris

2015

Stat Inference Stoch Process

View full text Add to dashboard Cite

This article addresses the problem of functional supervised classification of Cox process trajectories, whose random intensity is driven by some exogenous random covariable. The classification task is achieved through a regularized convex empirical risk minimization procedure, and a nonasymptotic oracle inequality is derived. We show that the algorithm provides a Bayes-risk consistent classifier. Furthermore, it is proved that the classifier converges at a rate which adapts to the unknown regularity of the intensity process. Our results are obtained by taking advantage of martingale and stochastic calculus arguments, which are natural in this context and fully exploit the functional nature of the problem.

show abstract

“…11 The monotonized estimates are obtained using the average rearrangement over both the price 9 There is potentially a large set of methods that can be used to choose the number of series terms (crossvalidation, penalization, the method of Lepski, among others). Indeed, the problem of selecting the number of series terms is a special case of the problem of model selection, and there are several textbooks/monographs in the literature on model selection in abstract settings; for example, Massart [65] and Koltchinskii [56].…”

Section: Examplesmentioning

confidence: 99%

Conditional quantile processes based on series or many regressors

Belloni

Chernozhukov

Chetverikov

et al. 2011

View full text Add to dashboard Cite

Abstract. Quantile regression (QR) is a principal regression method for analyzing the impact of covariates on outcomes. The impact is described by the conditional quantile function and its functionals. In this paper we develop the nonparametric QR-series framework, covering many regressors as a special case, for performing inference on the entire conditional quantile function and its linear functionals. In this framework, we approximate the entire conditional quantile function by a linear combination of series terms with quantile-specific coefficients and estimate the function-valued coefficients from the data.We develop large sample theory for the QR-series coefficient process, namely we obtain uniform strong approximations to the QR-series coefficient process by conditionally pivotal and Gaussian processes. Based on these two strong approximations, or couplings, we develop four resampling methods (pivotal, gradient bootstrap, Gaussian, and weighted bootstrap) that can be used for inference on the entire QR-series coefficient function.We apply these results to obtain estimation and inference methods for linear functionals of the conditional quantile function, such as the conditional quantile function itself, its partial derivatives, average partial derivatives, and conditional average partial derivatives.Specifically, we obtain uniform rates of convergence and show how to use the four resampling methods mentioned above for inference on the functionals. All of the above results are for function-valued parameters, holding uniformly in both the quantile index and the covariate value, and covering the pointwise case as a by-product. We demonstrate the practical utility of these results with an empirical example, where we estimate the price elasticity function and test the Slutsky condition of the individual demand for gasoline, as indexed by the individual unobserved propensity for gasoline consumption.Date: first version May, 2006, this version of August 1, 2017. The main results of this paper, particularly the pivotal method for inference based on the entire quantile regression process, were first presented at the NAWM Econometric Society, New Orleans, January, 2008 and also at the Stats in the Chateau in September 2009. We are grateful to Arun Chandraksekhar, Ye Luo, Denis Tkachenko, and Sami Stouli for careful readings of several versions of the paper. We thank Gary Chamberlain, Andrew Chesher, Holger Dette, Roger Koenker, Tatiana Komarova, Arthur Lewbel, Oliver Linton, Whitney Newey, Zhongjun Qu, and seminar participants at the Econometric Society meeting, CEME Econometrics of Demand conference, CEMMAP master-class, CIREQ High-Dimensional Problems in Econometrics conference, ERCIM conference, ISI World Statistics Congress, Oberwolfach Frontiers in Quantile Regression workshop, Stats in the Chateau, Austin, BU, CEMFI, Columbia, Duke, Harvard/MIT, NUS, Rutgers, Sciences Po, SMU, Upenn, Virginia, and Yale for many useful suggestions. We are grateful to Adonis Yatchew for giving us permission to use the data set i...

show abstract

Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems

Abstract: The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

Cited by 357 publications

References 105 publications

An Introduction to Matrix Concentration Inequalities

An Introduction to Matrix Concentration Inequalities

Cox process functional learning

Conditional quantile processes based on series or many regressors

Contact Info

Product

Resources

About