Fast Cross-validation for Multi-penalty High-dimensional Ridge Regression

Wiel, Mark A. van de; Nee, Mirrelijn M. van; Rauschenberger, Armin

doi:10.1080/10618600.2021.1904962

Cited by 19 publications

(21 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…with X i and Z k the i th and k th row of X and Z respectively, γ (d) ∈ R G the co-data variable weights for co-data matrix d, w the co-data matrix weights and τ 2 global a scaling factor which may improve numerical computations in practice. When the data X consist of multiple data modalities, like gene expression data, copy number data and methylation data in genomics, scaling factors specific to the data modalities may be used (Boulesteix et al, 2017;van de Wiel et al, 2021) and estimated with ecpc.…”

Section: Methodsmentioning

confidence: 99%

ecpc: An R-package for generic co-data models for high-dimensional prediction

Nee¹,

Wessels²,

Wiel³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

High-dimensional prediction considers data with more variables than samples. Generic research goals are to find the best predictor or to select variables. Results may be improved by exploiting prior information in the form of co-data, providing complementary data not on the samples, but on the variables. We consider adaptive ridge penalised generalised linear and Cox models, in which the variable specific ridge penalties are adapted to the co-data to give a priori more weight to more important variables. The R-package ecpc originally accommodated various and possibly multiple co-data sources, including categorical co-data, i.e. groups of variables, and continuous co-data. Continuous co-data, however, was handled by adaptive discretisation, potentially inefficiently modelling and losing information. Here, we present an extension to the method and software for generic co-data models, particularly for continuous co-data. At the basis lies a classical linear regression model, regressing prior variance weights on the co-data. Co-data variables are then estimated with empirical Bayes moment estimation. After placing the estimation procedure in the classical regression framework, extension to generalised additive and shape constrained co-data models is straightforward. Besides, we show how ridge penalties may be transformed to elastic net penalties with the R-package squeezy. In simulation studies we first compare various co-data models for continuous co-data from the extension to the original method. Secondly, we compare variable selection performance to other variable selection methods. Moreover, we demonstrate use of the package in several examples throughout the paper. * The first author is supported by ZonMw TOP grant COMPUTE CANCER (40-00812-98-16012).

show abstract

Section: Methodsmentioning

confidence: 99%

ecpc: An R-package for generic co-data models for high-dimensional prediction

Nee¹,

Wessels²,

Wiel³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Banded ridge regression has been proposed multiple times in the scientific literature, either with a Bayesian approach ( Ignatiadis and Lolas, 2020 ; van Nee et al, 2021 ; Perrakis et al, 2020 ) or with a frequentist approach ( Nunez-Elizalde et al, 2019 ; van de Wiel et al, 2021 ). In the Bayesian approach, a prior assumption is made on the regularization hyperparameter distribution to simplify the optimization problem.…”

Section: Banded Ridge Regression and Feature-space Selectionmentioning

confidence: 99%

Feature-space selection with banded ridge regression

et al. 2022

View full text Add to dashboard Cite

“…Banded ridge regression has been proposed multiple times in the scientific literature, either with a Bayesian approach [Perrakis et al, 2020, Ignatiadis and Lolas, 2020, van Nee et al, 2021] or with a frequentist approach [Nunez-Elizalde et al, 2019, van de Wiel et al, 2021]. In the Bayesian approach, a prior assumption is made on the regularization hyperparameter distribution to simplify the optimization problem.…”

Section: Banded Ridge Regression and Feature-space Selectionmentioning

confidence: 99%

Feature-space selection with banded ridge regression

Tour

Eickenberg

Núñez-Elizalde

et al. 2022

Preprint

View full text Add to dashboard Cite

Encoding models provide a powerful framework to identify the information represented in brain recordings. In this framework, a stimulus representation is expressed as a feature space and is used in a regularized linear regression to predict brain activity. To account for a potential complementarity of different feature spaces, a joint model is fit on multiple feature spaces simultaneously. To adapt regularization strength to each feature space, ridge regression is extended to banded ridge regression, which optimizes a different regularization hyperparameter per feature space. The present paper proposes a method to decompose over feature spaces the variance explained by a banded ridge regression model. It also describes how banded ridge regression performs a feature-space selection, effectively ignoring non-predictive and redundant feature spaces. This feature-space selection leads to better prediction accuracy and to better interpretability. Banded ridge regression is then mathematically linked to a number of other regression methods with similar feature-space selection mechanisms. Finally, several methods are proposed to address the computational challenge of fitting banded ridge regressions on large numbers of voxels and feature spaces. All implementations are released in an open-source Python package called "Himalaya".

show abstract

Fast Cross-validation for Multi-penalty High-dimensional Ridge Regression

Cited by 19 publications

References 30 publications

ecpc: An R-package for generic co-data models for high-dimensional prediction

ecpc: An R-package for generic co-data models for high-dimensional prediction

Feature-space selection with banded ridge regression

Feature-space selection with banded ridge regression

Contact Info

Product

Resources

About