Regularization of Wavelet Approximations

Antoniadis, Anestis; Fan, Jianqing

doi:10.1198/016214501753208942

Cited by 431 publications

(435 citation statements)

References 52 publications

Supporting

Mentioning

428

Contrasting

Unclassified

Order By: Relevance

“…The regression model (4.8) is a p > n sparsity problem. It is also a wavelet approximation problem that Antoniadis & Fan (2001) studied in the context of wavelet denoising, because…”

Section: θ-Ipod and Tispmentioning

confidence: 99%

See 1 more Smart Citation

Outlier Detection Using Nonconvex Penalized Regression

She

Owen

2011

Journal of the American Statistical Association

270

266

View full text Add to dashboard Cite

This paper studies the outlier detection problem from the point of view of penalized regressions. Our regression model adds one mean shift parameter for each of the n data points. We then apply a regularization favoring a sparse vector of mean shift parameters. The usual L 1 penalty yields a convex criterion, but we find that it fails to deliver a robust estimator. The L 1 penalty corresponds to soft thresholding. We introduce a thresholding (denoted by Θ) based iterative procedure for outlier detection (Θ-IPOD). A version based on hard thresholding correctly identifies outliers on some hard test problems. We find that Θ-IPOD is much faster than iteratively reweighted least squares for large data because each iteration costs at most O(np) (and sometimes much less) avoiding an O(np 2 ) least squares estimate. We describe the connection between Θ-IPOD and M -estimators. Our proposed method has one tuning parameter with which to both identify outliers and estimate regression coefficients. A data-dependent choice can be made based on BIC. The tuned Θ-IPOD shows outstanding performance in identifying outliers in various situations in comparison to other existing approaches. This methodology extends to highdimensional modeling with p ≫ n, if both the coefficient vector and the outlier pattern are sparse.

show abstract

“…The regression model (4.8) is a p > n sparsity problem. It is also a wavelet approximation problem that Antoniadis & Fan (2001) studied in the context of wavelet denoising, because…”

Section: θ-Ipod and Tispmentioning

confidence: 99%

“…Furthermore, the regularized one-step estimator (ROSE) (Antoniadis & Fan 2001) is the first step of Θ-IPOD.…”

Section: θ-Ipod and Tispmentioning

confidence: 99%

Outlier Detection Using Nonconvex Penalized Regression

She

Owen

2011

Journal of the American Statistical Association

270

266

View full text Add to dashboard Cite

show abstract

“…As a good starting point we recommend Huang (2003), Antoniadis and Fan (2001) and van de Geer (2001).…”

Section: Previous Workmentioning

confidence: 99%

Smooth functions and local extreme values

Kovac

2007

Computational Statistics & Data Analysis

View full text Add to dashboard Cite

Given a sample of n observations y 1 , . . . , y n at time points t 1 , . . . , t n we consider the problem of specifying a functionf such thatf• is smooth,• fits the data in the sense that the residuals y i −f(t i ) satisfy the multiresolution criterion• is as simple as possible so thatf exhibits the minimum number of local extreme values.We analyse in particular a fast method which is based on minimisingwhere the λ i are chosen automatically. The new method can also be applied to density estimation.

show abstract

“…So the selection of important variables corresponds to the selection of groups of basis functions. Methods for selecting grouped variables have been studied in Antoniadis and Fan (2001) and Yuan and Lin (2006). Lin and Zhang (2006) proposed the component selection and smoothing operator (COSSO) method for model selection in smoothing spline ANOVA with a fixed number of covariates.…”

Section: Introductionmentioning

confidence: 99%

Variable selection for sparse high-dimensional nonlinear regression models by combining nonnegative garrote and sure independence screening

Han

et al. 2014

STAT SINICA

View full text Add to dashboard Cite

In many regression problems, the relations between the covariates and the response may be nonlinear. Motivated by the application of reconstructing a gene regulatory network, we consider a sparse high-dimensional additive model with the additive components being some known nonlinear functions with unknown parameters. To identify the subset of important covariates, we propose a new method for simultaneous variable selection and parameter estimation by iteratively combining a large-scale variable screening (the nonlinear independence screening, NLIS) and a moderate-scale model selection (the nonnegative garrote, NNG) for the nonlinear additive regressions. We have shown that the NLIS procedure possesses the sure screening property and it is able to handle problems with non-polynomial dimensionality; and for finite dimension problems, the NNG for the nonlinear additive regressions has selection consistency for the unimportant covariates and also estimation consistency for the parameter estimates of the important covariates. The proposed method is applied to simulated data and a real data example for identifying gene regulations to illustrate its numerical performance.

show abstract

Regularization of Wavelet Approximations

Cited by 431 publications

References 52 publications

Outlier Detection Using Nonconvex Penalized Regression

Outlier Detection Using Nonconvex Penalized Regression

Smooth functions and local extreme values

Variable selection for sparse high-dimensional nonlinear regression models by combining nonnegative garrote and sure independence screening

Contact Info

Product

Resources

About