A Model-averaging method for high-dimensional regression with missing responses at random

Abstract

This article considers the ultrahigh-dimensional prediction problem in the presence of missing responses at random. A two-step model averaging procedure is proposed to improve prediction accuracy of conditional mean of response variable. The first step is to specify several candidate models, each with low-dimensional predictors. To implement this step, a new feature screening method is developed to distinguish from the active and inactive predictors via the multiple-imputation sure independence screening (MI-SIS) procedure, and candidate models are formed by grouping covariates with similar size of MI-SIS values. The second step is to develop a new criterion to find the optimal weights for averaging a set of candidate models via the weighted delete-one cross-validation (WDCV). Under some regularity conditions, we show that the proposed new screening statistic enjoys ranking consistency property, and the WDCV criterion asymptotically achieves the lowest possible prediction loss. Simulation studies and an example are illustrated by the proposed methodologies.

A robust model averaging approach for partially linear models with responses missing at random

“…Condition (C.1) is just a commonly used condition in nonparametric regression problem. Condition (C.2) is a standard condition in missing data literature (see, Condition (C.P) in Wang & Rao, 2002 and assumption 1 in Xie et al, 2021). Condition (C.3) is just a restriction conditional moments of 𝜖, which requires the distribution of 𝜖 to be sufficiently stable.…”

“…By combing the inverse probability weighting method and the $$$ {\mathrm{HRC}}_p $$$ approach of Liu and Okui (2013), they proved their method is asymptotically optimal with certain conditions. Recently, Wei and Wang (2021) and Xie et al (2021) established the cross‐validation model averaging criterion for linear models and high‐dimensional linear models by handling MAR responses in a similar way to Wei et al (2021), respectively. Correspondingly, they also separately obtained the asymptotic optimality of their methods.…”

High‐dimensional model averaging for quantile regression

“…More recently, Ando & Li (2014) proposed a two‐step model averaging procedure to find the optimal weights for UHD linear regression models using the delete‐one cross‐validation procedure. Subsequently, their idea was extended to linear models with incomplete data, generalized linear models, and quantile regression: see Ando & Li (2017), Xie et al (2021), Yan et al (2021), and Wang et al (2021). Chen et al (2018) provided two semiparametric UHD model averaging procedures for nonlinear dynamic time series regression models.…”