Sparse Additive Models

Ravikumar, Pradeep; Lafferty, John; Liu, Han; Wasserman, Larry

doi:10.1111/j.1467-9868.2009.00718.x

Cited by 489 publications

(541 citation statements)

References 21 publications

Supporting

Mentioning

524

Contrasting

Unclassified

Order By: Relevance

“…Lin and Zhang (2006) proposed the component selection and smoothing operator (COSSO) method for model selection in smoothing spline ANOVA with a fixed number of covariates. Recently, Ravikumar et al (2009), Meier et al (2009) and Huang et al (2010) considered variable selection in high dimensional nonparametric additive models where the number of additive components is larger than the sample size.…”

Section: Introductionmentioning

confidence: 99%

Variable selection for sparse high-dimensional nonlinear regression models by combining nonnegative garrote and sure independence screening

Han

et al. 2014

STAT SINICA

View full text Add to dashboard Cite

In many regression problems, the relations between the covariates and the response may be nonlinear. Motivated by the application of reconstructing a gene regulatory network, we consider a sparse high-dimensional additive model with the additive components being some known nonlinear functions with unknown parameters. To identify the subset of important covariates, we propose a new method for simultaneous variable selection and parameter estimation by iteratively combining a large-scale variable screening (the nonlinear independence screening, NLIS) and a moderate-scale model selection (the nonnegative garrote, NNG) for the nonlinear additive regressions. We have shown that the NLIS procedure possesses the sure screening property and it is able to handle problems with non-polynomial dimensionality; and for finite dimension problems, the NNG for the nonlinear additive regressions has selection consistency for the unimportant covariates and also estimation consistency for the parameter estimates of the important covariates. The proposed method is applied to simulated data and a real data example for identifying gene regulations to illustrate its numerical performance.

show abstract

Section: Introductionmentioning

confidence: 99%

Variable selection for sparse high-dimensional nonlinear regression models by combining nonnegative garrote and sure independence screening

Han

et al. 2014

STAT SINICA

View full text Add to dashboard Cite

show abstract

“…However, real data is not always Gaussian, and such an assumption can be limiting, especially when analyzing multiple data sources simultaneously, since non-Gaussianity in a single data source will result in the non-Gaussianity of the combined data. A lot of previous work has been done to drop the Gaussianity assumption in the solution to classic problems like sparse regression (Ravikumar et al, 2007), estimating GGMs (Liu et al, 2009), sparse CCA (Balakrishnan et al, 2012), etc., and propose nonparametric solutions to the same. We will also drop the assumption that the data is drawn from the same Gaussian distribution in the next section.…”

Section: Joint Estimation Of the Ggmmentioning

confidence: 99%

NP-MuScL: Unsupervised Global Prediction of Interaction Networks from Multiple Data Sources

Puniyani

Xing

2013

Journal of Computational Biology

View full text Add to dashboard Cite

Inference of gene interaction networks from expression data usually focuses on either supervised or unsupervised edge prediction from a single data source. However, in many real world applications, multiple data sources, such as microarray and ISH (in situ hybridization) measurements of mRNA abundances, are available to offer multiview information about the same set of genes. We propose ISH to estimate a gene interaction network that is consistent with such multiple data sources, which are expected to reflect the same underlying relationships between the genes. NP-MuScL casts the network estimation problem as estimating the structure of a sparse undirected graphical model. We use the semiparametric Gaussian copula to model the distribution of the different data sources, with the different copulas sharing the same precision (i.e., inverse covariance) matrix, and we present an efficient algorithm to estimate such a model in the high-dimensional scenario. Results are reported on synthetic data, where NP-MuScL outperforms baseline algorithms significantly, even in the presence of noisy data sources. Experiments are also run on two real-world scenarios: two yeast microarray datasets and three Drosophila embryonic gene expression datasets, where NP-MuScL predicts a higher number of known gene interactions than existing techniques.

show abstract

“…Instead, we seek a model that accommodates uncountably many predictors from the outset. Sparsity in the additive components, as considered for example in Ravikumar et al (2009) for additive modeling in the large p case, is not particularly meaningful in the context of a functional predictor. A more natural approach to overcome the inherent curse of dimensionality is to replace sparsity by continuity when predictors are smooth infinite-dimensional random trajectories, as considered here.…”

Section: Introductionmentioning

confidence: 99%

Continuously additive models for nonlinear functional regression

Müller

Yao

2013

Biometrika

View full text Add to dashboard Cite

SummaryWe introduce continuously additive models, which can be motivated as extensions of additive regression models with vector predictors to the case of infinite-dimensional predictors.This approach provides a class of flexible functional nonlinear regression models, where random predictor curves are coupled with scalar responses. In continuously additive modeling, integrals taken over a smooth surface along graphs of predictor functions relate the predictors to the responses in a nonlinear fashion. We use tensor product basis expansions to fit the smooth regression surface that characterizes the model. In a theoretical investigation, we show that the predictions obtained from fitting continuously additive models are consistent and asymptotically normal. We also consider extensions to generalized responses. The proposed approach outperforms existing functional regression models in simulations and data illustrations.

show abstract

Sparse Additive Models

Cited by 489 publications

References 21 publications

Variable selection for sparse high-dimensional nonlinear regression models by combining nonnegative garrote and sure independence screening

Variable selection for sparse high-dimensional nonlinear regression models by combining nonnegative garrote and sure independence screening

NP-MuScL: Unsupervised Global Prediction of Interaction Networks from Multiple Data Sources

Continuously additive models for nonlinear functional regression

Contact Info

Product

Resources

About