Fridge: Focused fine‐tuning of ridge regression for personalized predictions

Abstract

Statistical prediction methods typically require some form of fine-tuning of tuning parameter(s), with K-fold cross-validation as the canonical procedure. For ridge regression, there exist numerous procedures, but common for all, including cross-validation, is that one single parameter is chosen for all future predictions. We propose instead to calculate a unique tuning parameter for each individual for which we wish to predict an outcome. This generates an individualized prediction by focusing on the vector of covariates of a specific individual. The focused ridge-fridge-procedure is introduced with a 2-part contribution: First we define an oracle tuning parameter minimizing the mean squared prediction error of a specific covariate vector, and then we propose to estimate this tuning parameter by using plug-in estimates of the regression coefficients and error variance parameter. The procedure is extended to logistic ridge regression by using parametric bootstrap. For high-dimensional data, we propose to use ridge regression with cross-validation as the plug-in estimate, and simulations show that fridge gives smaller average prediction error than ridge with cross-validation for both simulated and real data. We illustrate the new concept for both linear and logistic regression models in 2 applications of personalized medicine: predicting individual risk and treatment response based on gene expression data. The method is implemented in the R package fridge.

Estimation of variance components, heritability and the ridge penalty in high-dimensional generalized linear models

“…Here, the residual effective degrees of freedom [16] equals ν = n − tr(2H − HH T ), with H as in (2). We also considered (9) with ν = n − tr(H), as in [17], which rendered similar, slightly inferior results.…”

“…Such a prediction can be used to decide upon additional measures to prevent excessive weight gains. We reproduced the analysis by [17] as much as possible, including their prior selection of 1000 genes. Details on minor discrepancies, and an alternative analysis that accounts for the gene selection are discussed in the Supplementary Material.…”

“…In [17], the authors illustrate their focused ridge (fridge) method and compare it with conventional ridge. In short, fridge estimates sample-specific ridge penalties, based on minimizing a per sample mean squared error (MSE) criterion on the level of the linear predictor X i β.…”

“…In the main document we re-analysed the weight gain data recently discussed in [17]. The authors kindly provided the R-code corresponding to the results in [17], which includes their focused ridge (fridge) methodology. Below we report a small discrepancy between our analysis and theirs, and also present results of an alternative analysis.…”

“…The authors of [17] opted to present results on n = 25 (rather than n = 26) samples, for technical reasons (personal communication). Results differed very little, so we chose to use all n = 26 samples of the original study [20].…”