Simulation-Selection-Extrapolation: Estimation in High-Dimensional Errors-in-Variables Models

Abstract

Errors-in-variables models in high-dimensional settings pose two challenges in application. First, the number of observed covariates is larger than the sample size, while only a small number of covariates are true predictors under an assumption of model sparsity. Second, the presence of measurement error can result in severely biased parameter estimates, and also affects the ability of penalized methods such as the lasso to recover the true sparsity pattern. A new estimation procedure called SIMulation-SELection-EXtrapolation (SIMSELEX) is proposed. This procedure makes double use of lasso methodology. First, the lasso is used to estimate sparse solutions in the simulation step, after which a group lasso is implemented to do variable selection. The SIMSELEX estimator is shown to perform well in variable selection, and has significantly lower estimation error than naive estimators that ignore measurement error. SIMSELEX can be applied in a variety of errors-in-variables settings, including linear models, generalized linear models, and Cox survival models. It is furthermore shown in the Supporting Information how SIMSELEX can be applied to spline-based regression models. A simulation study is conducted to compare the SIMSELEX estimators to existing methods in the linear and logistic model settings, and to evaluate performance compared to naive methods in the Cox and spline models. Finally, the method is used to analyze a microarray dataset that contains gene expression measurements of favorable histology Wilms tumors.

Screening methods for linear errors-in-variables models in high dimensions

“…We followed the procedures described in Sørensen et al (2015), Nghiem and Potgieter (2019) and Romeo and Thoresen (2019) to process the raw data using the BGX package of Hein et al (2005), and assumed the measurement error on each gene was mutually independent from that on the other. As a result, the measurement error covariance matrix Σ u was set to be diagonal.…”

“…The gene expression measurements tend to be noisy, where measurement errors come from many sources such as sample preparation, labeling, and hybridization; for example, see Rocke and Durbin (2001) and Zakharkin et al (2005). The gene measurements are also often analyzed on the log scale, making the assumption of additive measurement errors more plausible (Nghiem and Potgieter, 2019). Furthermore, as in common genome wide association studies (Do et al, 2011;Zhou et al, 2018, among others), it is usually assumed that only a few genes are related to the outcome of interest, i.e a sparsity assumption on the statistical model.…”

“…Brown et al (2019) introduced a boosting algorithm based on the estimating equation of the corrected lasso, but the theoretical properties of the final estimates were not examined. In another line of research, Nghiem and Potgieter (2019) proposed a SIMSELEX estimator that first uses simulation to evaluate the effect of measurement error on estimated coefficients, then selects important covariates based on these simulated effects, and finally extrapolates the simulation to the scenario with no measurement error present. Furthermore, Byrd and McGee (2019) presented an EM-type correction method, where they iteratively sample the true covariate from the conditional distribution of X given y and W, and fitted penalized regressions of y on these sampled covariates.…”

Classifiers for Predicting Coronary Artery Disease Based on Gene Expression Profiles in Peripheral Blood Mononuclear Cells

“…Combinations of biomarkers have been integrated using various methods in order to create diagnostic or prognostic tools in CAD. These methods include the least absolute shrinkage and selection operator (LASSO), 10 random forest (RF) classifier, 11 support vector machine (SVM), 12 and the gene set variation analysis (GSVA) score. 13 LASSO is a commonly used penalty regression method, which can be applied for selection of variables in high-dimensional data.…”

“…13 LASSO is a commonly used penalty regression method, which can be applied for selection of variables in high-dimensional data. 10 LASSO performs via a continuous shrinking operation, minimizing regression coefficients in order to reduce the likelihood of overfitting. 14 RF is a model that can deal with unbalanced sample distribution, generating less biased classifiers, 11 but it often fails to be robust and is vulnerable to overfitting.…”