Sparse classification with paired covariates

Abstract

This paper introduces the paired lasso: a generalisation of the lasso for paired covariate settings. Our aim is to predict a single response from two high-dimensional covariate sets. We assume a one-to-one correspondence between the covariate sets, with each covariate in one set forming a pair with a covariate in the other set. Paired covariates arise, for example, when two transformations of the same data are available. It is often unknown which of the two covariate sets leads to better predictions, or whether the two covariate sets complement each other. The paired lasso addresses this problem by weighting the covariates to improve the selection from the covariate sets and the covariate pairs. It thereby combines information from both covariate sets and accounts for the paired structure. We tested the paired lasso on more than 2000 classification problems with experimental genomics data, and found that for estimating sparse but predictive models, the paired lasso outperforms the standard and the adaptive lasso. The R package is available from cran.

Ten quick tips for biomarker discovery and validation analyses using machine learning

“…Finally, for many biomedical applications, natural structures among features or complementary information on the features can be exploited as an additional information source for model building. For example, among causally related features, we might want to prioritize the selection of upstream over downstream features in a known causal graph [ 100 ] to account for pairs or groups of functionally related features [ 101 , 102 ] or to transfer information from previous studies (i.e., prior weights or prior effects) into the learning procedure. These approaches to integrate prior knowledge into the learning phase have the potential to render models more predictive and more interpretable.…”

BLADE: Bayesian Log-normAl DEconvolution for enhanced in silico microdissection of bulk gene expression data

“…We considered all complete samples from the following tumor types: Mesothelioma (MESO 31 , n = 84; and Sarcoma (SARC 32 , n = 256. Data was preprocessed as described previously in Rauschenberger et al 33 . The comparison procedure for LN and NB distributions is:…”

Fast cross-validation for multi-penalty ridge regression

“…Other (large) sets such as breast cancer and ovarian cancer (BRAC and OV) rendered no or a weak signal at best (c-index < 0.6) for any of the methods below, and hence these results were not shown. The data were preprocessed as described in Rauschenberger et al (2019). Further details on the data are given in the Appendix.…”

“…As in ordinary regression settings, the best scale to represent a given data type is not known beforehand. In fact, in an adaptive elastic net setting, the joint use of a continuous and binary representation was shown to be potentially beneficial (Rauschenberger et al, 2019) for omics-based tumor classification. Our default multi-ridge (multiridge) allows for including both representations using different penalties to reflect different predictive signal for the two representations.…”

“…• Thymoma (THYM; Cancer Genome Atlas Research Network et al, 2018a), n = 118 Data was preprocessed as described previously in Rauschenberger et al (2019). x-axis) versus crossvalidated performance (log-likelihood, AUC; y-axis), using either one or five repeats of 5-fold CV.…”