Deselection of base-learners for statistical boosting—with an application to distributional regression

Strömer, Annika; Staerk, Christian; Klein, Nadja; Weinhold, Leonie; Titze, Stephanie; Mayr, Andreas

doi:10.1177/09622802211051088

Cited by 13 publications

(19 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to classical regression methods, boosting does not provide closed formulas for standard errors of effect estimates or confidence intervals April 29, 2022 14/23 that could be used for inference. Furthermore, as mentioned before, statistical boosting is in general associated with a slightly higher computational complexity compared to methods such as the lasso [26] and has a known tendency to include too many variables in low-dimensional settings [19,45]. Our results suggest that the incorporation of batches substantially reduced the computational time.…”

Section: Discussionsupporting

confidence: 57%

Boosting polygenic risk scores

Klinkhammer

Staerk

Maj

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Polygenic risk scores (PRS) evaluate the individual genetic liability to a certain trait and are expected to play an increasingly important role in the field of clinical risk stratification. Most often, PRS are estimated based on summary statistics of univariate effects derived from genome-wide association studies. To improve the predictive performance of PRS, it is desirable to fit multivariable models directly on the genetic data. Due to the large and high-dimensional data, a direct application of existing methods is often not feasible and new efficient algorithms are required to overcome the computational burden regarding efficiency and memory demands.We develop an adapted component-wise L2-boosting algorithm to fit genotype data from large cohort studies to continuous outcomes using linear base-learners for the genetic variants. Similar to the snpnet approach implementing lasso regression, the proposed snpboost approach iteratively works on smaller batches of variants. By restricting the set of possible base-learners in each boosting step to variants most correlated with the residuals from previous iterations, the computational efficiency can be substantially increased without losing prediction accuracy. Furthermore, for large-scale data based on various traits from the UK Biobank we show that our method yields competitive prediction accuracy and computational efficiency compared to the snpnet approach. Due to the modular structure of boosting, our framework can be further extended to construct PRS for different outcome data and effect types.

show abstract

Section: Discussionsupporting

confidence: 57%

Boosting polygenic risk scores

Klinkhammer

Staerk

Maj

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In contrast to classical regression methods, boosting does not provide closed formulas for standard errors of effect estimates or confidence intervals that could be used for inference. Furthermore, as mentioned before, statistical boosting is in general associated with a slightly higher computational complexity compared to methods such as the lasso ( Hepp et al, 2016 ) and has a known tendency to include too many variables in low-dimensional settings ( Staerk and Mayr, 2021 ; Strömer et al, 2022 ). Our results suggest that the incorporation of batches substantially reduced the computational time.…”

Section: Discussionmentioning

confidence: 99%

A statistical boosting framework for polygenic risk scores based on large-scale genotype data

et al. 2023

Self Cite

View full text Add to dashboard Cite

Polygenic risk scores (PRS) evaluate the individual genetic liability to a certain trait and are expected to play an increasingly important role in clinical risk stratification. Most often, PRS are estimated based on summary statistics of univariate effects derived from genome-wide association studies. To improve the predictive performance of PRS, it is desirable to fit multivariable models directly on the genetic data. Due to the large and high-dimensional data, a direct application of existing methods is often not feasible and new efficient algorithms are required to overcome the computational burden regarding efficiency and memory demands. We develop an adapted component-wise L2-boosting algorithm to fit genotype data from large cohort studies to continuous outcomes using linear base-learners for the genetic variants. Similar to the snpnet approach implementing lasso regression, the proposed snpboost approach iteratively works on smaller batches of variants. By restricting the set of possible base-learners in each boosting step to variants most correlated with the residuals from previous iterations, the computational efficiency can be substantially increased without losing prediction accuracy. Furthermore, for large-scale data based on various traits from the UK Biobank we show that our method yields competitive prediction accuracy and computational efficiency compared to the snpnet approach and further commonly used methods. Due to the modular structure of boosting, our framework can be further extended to construct PRS for different outcome data and effect types—we illustrate this for the prediction of binary traits.

show abstract

“…A limitation of our algorithm is the relatively high selection rates for variables with only minor importance, which occurs particularly in low‐dimensional settings. In this context, Strömer et al 66 have recently proposed an approach to deselect predictors with negligible impact to obtain sparser models with statistical boosting. We want to investigate the incorporation of this proposal in the context of multivariate GAMLSS in the future.…”

Section: Discussionmentioning

confidence: 99%

Boosting multivariate structured additive distributional regression models

Strömer

Klein

Staerk

et al. 2023

Statistics in Medicine

Self Cite

View full text Add to dashboard Cite

We develop a model‐based boosting approach for multivariate distributional regression within the framework of generalized additive models for location, scale, and shape. Our approach enables the simultaneous modeling of all distribution parameters of an arbitrary parametric distribution of a multivariate response conditional on explanatory variables, while being applicable to potentially high‐dimensional data. Moreover, the boosting algorithm incorporates data‐driven variable selection, taking various different types of effects into account. As a special merit of our approach, it allows for modeling the association between multiple continuous or discrete outcomes through the relevant covariates. After a detailed simulation study investigating estimation and prediction performance, we demonstrate the full flexibility of our approach in three diverse biomedical applications. The first is based on high‐dimensional genomic cohort data from the UK Biobank, considering a bivariate binary response (chronic ischemic heart disease and high cholesterol). Here, we are able to identify genetic variants that are informative for the association between cholesterol and heart disease. The second application considers the demand for health care in Australia with the number of consultations and the number of prescribed medications as a bivariate count response. The third application analyses two dimensions of childhood undernutrition in Nigeria as a bivariate response and we find that the correlation between the two undernutrition scores is considerably different depending on the child's age and the region the child lives in.

show abstract

Deselection of base-learners for statistical boosting—with an application to distributional regression

Cited by 13 publications

References 44 publications

Boosting polygenic risk scores

Boosting polygenic risk scores

A statistical boosting framework for polygenic risk scores based on large-scale genotype data

Boosting multivariate structured additive distributional regression models

Contact Info

Product

Resources

About