Robust adaptive Lasso in high-dimensional logistic regression with an application to genomic classification of cancer patients

Abstract

Penalized logistic regression is extremely useful for binary classification with a large number of covariates (significantly higher than the sample size), having several real life applications, including genomic disease classification. However, the existing methods based on the likelihood based loss function are sensitive to data contamination and other noise and, hence, robust methods are needed for stable and more accurate inference. In this paper, we propose a family of robust estimators for sparse logistic models utilizing the popular density power divergence based loss function and the general adaptively weighted LASSO penalties. We study the local robustness of the proposed estimators through its influence function and also derive its oracle properties and asymptotic distribution. With extensive empirical illustrations, we clearly demonstrate the significantly improved performance of our proposed estimators over the existing ones with particular gain in robustness. Our proposal is finally applied to analyse four different real datasets for cancer classification, obtaining robust and accurate models, that simultaneously performs gene selection and patient classification.Software availability: A ready implementation of our proposal is available as an R package awD-PDlasso in Github.

Classification of COVID19 Patients Using Robust Logistic Regression

“…The minimization of the objective (9) produces robust adaptively weighted DPD-LASSO estimators, which includes the DPD-LASSO estimator for w(•) = 1. The resulting penalized MDPDEs are indeed robust for all positives values of α when the initial estimator β is also robust, as proved in Basu et al [7], and non-robust at α = 0 corresponding to the MLE. Moreover, they are consistent and asymptotically normal in the high dimensional data set-up with non polynomial order, i.e., when log(k) = O(n s ) for some s ∈ (0, 1), under some regularity conditions.…”

“…Fokianos [11], Park and Hastie [21], Plan and Vershynin [23], Zhu and Hastie [30] and Sun and Wang [26] are interesting papers based on the LASSO estimator for the logistic regression model. Basu et al [7] extended the LASSO procedure with DPD-based loss function for the logistic regression model, producing more robust estimators. The so-called LASSO penalized MDPDE (DPD-LASSO) is then given by…”

“…The DPD has the interpretation of being a natural generalization of the likelihood-based loss function, so that the MLE is included as a particular case of the DPD-based family. Ghosh and Basu [13] studied the minimum DPD estimator (MDPDE) for generalized linear regression models, including the logistic regression, in the low-dimensional set-up, and later Basu et al [7] extended the methodology for the high-dimensional logistic regression model, yielding the penalized MDPDE. They considered several penalty functions, including the LASSO and the Ad-LASSO penalties, as well as more general weighted adaptive LASSO (AW-LASSO) penalties.…”

“…They considered several penalty functions, including the LASSO and the Ad-LASSO penalties, as well as more general weighted adaptive LASSO (AW-LASSO) penalties. This last work [7] is particularly motivated from the excellent performances, both in terms of estimation accuracy and variable selection optimality, of the penalized DPDbased procedures observed under the high-dimensional linear regression model [14,15].…”

“…In this paper, we propose to develop a COVID19 patients classifier through their upper airway gene expression using the penalized logistic regression model, which simultaneously carries out important gene selection. We apply different estimation methods, namely the classical MLE and the MDPDEs penalized with the LASSO, Ad-LASSO and AW-LASSO penalties (specific to the SCAD penalty of Fan and Li [10]) as developed in [7]. While adaptive penalties enhance the variable selection property, robust procedures based on the DPD have been shown to perform competitively with non-robust ones in the absence of contamination, and to improve the estimation accuracy and robustness in a contaminated scenario.…”