High-dimensional additive modeling

Meier, Lukas; Geer, Sara van de; Bühlmann, Peter

doi:10.1214/09-aos692

Cited by 353 publications

(422 citation statements)

References 30 publications

Supporting

Mentioning

411

Contrasting

Order By: Relevance

“…However, the original Lasso method has mainly been designed for regression models with a linear prediction function. Combining penalized estimation with sparse nonlinear additive modeling has only recently been accomplished (Meier et al 2009). To date, there is no extension of the method developed by Meier et al (2009) to geoadditive proportional odds models.…”

Section: Discussionmentioning

confidence: 99%

“…Combining penalized estimation with sparse nonlinear additive modeling has only recently been accomplished (Meier et al 2009). To date, there is no extension of the method developed by Meier et al (2009) to geoadditive proportional odds models. On the other hand, gradient boosting and the Lasso are closely related, as both algorithms can be embedded into the LARS framework (Efron et al 2004).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Geoadditive regression modeling of stream biological condition

et al. 2010

View full text Add to dashboard Cite

Indices of biotic integrity have become an established tool to quantify the condition of small non-tidal streams and their watersheds. To investigate the effects of watershed characteristics on stream biological condition, we present a new technique for regressing IBIs on watershed-specific explanatory variables. Since IBIs are typically evaluated on an ordinal scale, our method is based on the proportional odds model for ordinal outcomes. To avoid overfitting, we do not use classical maximum likelihood estimation but a component-wise functional gradient boosting approach. Because component-wise gradient boosting has an intrinsic mechanism for variable selection and model choice, determinants of biotic integrity can be identified. In addition, the method offers a relatively simple way to account for spatial correlation in ecological data. An analysis of the Maryland Biological Streams Survey shows that nonlinear effects of predictor variables on stream condition can be quantified while, in addition, accurate predictions of biological condition at unsurveyed locations are obtained.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Geoadditive regression modeling of stream biological condition

et al. 2010

View full text Add to dashboard Cite

show abstract

“…When p is fixed and small, the ordinary least squares estimators can be obtained for (2.3). When p is moderately high, penalized regression methods with grouped penalties have been well studied for (2.3), such as Lin and Zhang (2006), Meier, Geer and Bühlmann (2009) and Huang, Horowitz and Wei (2010). When p is much higher than the sample size, Fan, Feng and Song (2011) proposed a nonparametric independence screening (NIS) to reduce the dimensionality efficiently.…”

Section: Model Setupsmentioning

confidence: 99%

“…In the literature, penalized regression methods have been well studied for nonparametric additive models. See Lin and Zhang (2006), Meier, Geer andBühlmann (2009) andHuang, Horowitz andWei (2010). For sparse ultrahigh dimensional additive models, Fan, Feng and Song (2011) designed a nonparametric independence screening (NIS) which fits marginal nonparametric regressions of the response against each predictor individually and ranks their importance according to the magnitude of estimated nonparametric components.…”

Section: Introductionmentioning

confidence: 99%

Forward Additive Regression for Ultrahigh Dimensional Nonparametric Additive Models

Zhong¹,

Duan²,

Zhu³

2020

STAT SINICA

View full text Add to dashboard Cite

Ultrahigh dimensional data are collected in many scientific fields where the predictor dimension is often much higher than the sample size. To reduce the ultrahigh dimensionality effectively, many marginal screening approaches are developed. However, existing screening methods may miss some important predictors which are marginally independent of the response, or select some unimportant ones due to their high correlations with the important predictors. Iterative screening procedures are proposed to address this issue. However, studying their theoretical properties is not straightforward. Penalized regression are not computationally efficient or numerically stable when the predictors are ultrahigh dimensional. To overcome these drawbacks, Wang (2009) proposed a novel Forward Regression (FR) approach for linear models. However, nonlinear dependence between predictors and the response is often present in ultrahigh dimensional problems. In this paper, we further extend the FR to develop a Forward Additive Regression (FAR) method for selecting significant predictors in ultrahigh dimensional nonparametric additive models. We establish the screening consistency for the FAR method and examine its finite-sample performance by Monte Carlo simulations. Our simulations indicate that, compared with marginal screenings, the FAR is shown to be much more effective to identify important predictors for additive models. When the predictors are highly correlated, the FAR even performs better than the iterative marginal screenings, such as iterative nonparametric independence screening (INIS). We also apply the FAR method to a real data analysis in genetic studies.Key words and phrases: Additive models, forward regression, screening consistency, ultrahigh dimensionality, variable selection.Statistica Sinica: Newly accepted Paper (accepted author-version subject to English editing) 2 W. Zhong, S. Duan and L. Zhu IntroductionAdvances of modern information technology allow researchers in various scientific fields to collect high dimensional data where the number of predictors is greater than the sample size. Under the sparsity assumption that only a small subset of predictors truly contribute to the response, penalized regression methods have been intensively studied for various parametric and nonparametric models in the literature. They include, but are not limited to, LASSO (Tibshirani, 1996) (Candes and Tao, 2007). These methods are able to select significant variables and estimate parameters simultaneously. As a result, both model interpretability and predictability could be enhanced.When the predictor dimension is much greater than the sample size, the aforementioned penalized approaches may suffer from computational complexity, algorithmic instability or statistical inaccuracy (Fan, Samworth and Wu, 2009). Since the seminal work of Fan and Lv (2008), various marginal screening procedures have been proposed to reduce the ultrahigh dimensionality. The key idea of screening is to rank all predictors using a marginal util...

show abstract

“…Lin and Zhang (2006) proposed the component selection and smoothing operator (COSSO) method for model selection in smoothing spline ANOVA with a fixed number of covariates. Recently, Ravikumar et al (2009), Meier et al (2009) and Huang et al (2010) considered variable selection in high dimensional nonparametric additive models where the number of additive components is larger than the sample size.…”

Section: Introductionmentioning

confidence: 99%

Variable selection for sparse high-dimensional nonlinear regression models by combining nonnegative garrote and sure independence screening

Han

et al. 2014

STAT SINICA

View full text Add to dashboard Cite

In many regression problems, the relations between the covariates and the response may be nonlinear. Motivated by the application of reconstructing a gene regulatory network, we consider a sparse high-dimensional additive model with the additive components being some known nonlinear functions with unknown parameters. To identify the subset of important covariates, we propose a new method for simultaneous variable selection and parameter estimation by iteratively combining a large-scale variable screening (the nonlinear independence screening, NLIS) and a moderate-scale model selection (the nonnegative garrote, NNG) for the nonlinear additive regressions. We have shown that the NLIS procedure possesses the sure screening property and it is able to handle problems with non-polynomial dimensionality; and for finite dimension problems, the NNG for the nonlinear additive regressions has selection consistency for the unimportant covariates and also estimation consistency for the parameter estimates of the important covariates. The proposed method is applied to simulated data and a real data example for identifying gene regulations to illustrate its numerical performance.

show abstract

High-dimensional additive modeling

Cited by 353 publications

References 30 publications

Geoadditive regression modeling of stream biological condition

Geoadditive regression modeling of stream biological condition

Forward Additive Regression for Ultrahigh Dimensional Nonparametric Additive Models

Variable selection for sparse high-dimensional nonlinear regression models by combining nonnegative garrote and sure independence screening

Contact Info

Product

Resources

About