High-dimensional semi-supervised learning: in search for optimal inference of the mean

Abstract

We provide a high-dimensional semi-supervised inference framework focused on the mean and variance of the response. Our data are comprised of an extensive set of observations regarding the covariate vectors and a much smaller set of labeled observations where we observe both the response as well as the covariates. We allow the size of the covariates to be much larger than the sample size and impose weak conditions on a statistical form of the data. We provide new estimators of the mean and variance of the response that extend some of the recent results presented in low-dimensional models.In particular, at times we will not necessitate consistent estimation of the functional form of the data. Together with estimation of the population mean and variance, we provide their asymptotic distribution and confidence intervals where we showcase gains in efficiency compared to the sample mean and variance. Our procedure, with minor modifications, is then presented to make important contributions regarding inference about average treatment effects. We also investigate the robustness of estimation and coverage and showcase widespread applicability and generality of the proposed method.

On the role of surrogates in the efficient estimation of treatment effects with limited outcome data

“…This estimator δ is exactly identical to the estimator δ in Definition 1 under MCAR assumption. In Appendix A, we further discuss how this estimator connects to those in Cheng et al [2018], Zhang and Bradic [2019], and how our result generalizes those in previous literature.…”

“…Condition (3) requires the primary outcome to be missing at random (MAR), i.e., the indicator R depends on only observed variables, including pre-treatment covariates X, the treatment T and surrogates S. This condition guarantees that the distribution of the primary outcome on the labelled data and unlabelled data are comparable after accounting for the observed variables, so that we can use the labelled data to infer information about the missing primary outcome in the unlabelled data. This condition is considerably weaker than the missing completely at random (MCAR) condition typically assumed in previous semi-supervised inference literature [e.g., Cheng et al, 2018, Zhang andBradic, 2019], since MCAR does not allow the missingness of the primary outcome to depend on any other variable. Condition (3) may be satisfied by design in a two-phase sampling scheme [e.g., Wang et al, 2009, Cochran, 2007: in the first phase, relatively cheap measurements of T, X, S are available for all units, and in the second phase, expensive measurements of the primary outcome Y are collected for a validation subsample selected according to variables measured in the first phase.…”

“…This condition guarantees that the distribution of the subsample with primary-outcome observations (labelled data) is comparable with the distribution of the remaining sample without primary-outcome observations (unlabelled data) after adjusting for the observed variables. Similar missingness conditions are also commonly assumed in previous literature that combine different datasets [e.g., Athey et al, 2019, Cheng et al, 2018, Zhang and Bradic, 2019. Under only these standard assumptions, and in particular no overly restrictive surrogate conditions, we aim to investigate the role of surrogates in estimating treatment effects when the primary-outcome observations are limited.…”

“…Our paper is related to the growing body of literature on parameter estimation and inference in the semi-supervised setting where a small labelled dataset is enriched with a large unlabelled dataset. A string of research investigate how to the use unlabelled data to aid the estimation of a wide variety of parameters, including linear regression coefficients [Azriel et al, 2016, Chakrabortty et al, 2018, population mean and average treatment effect Bradic, 2019], performance measures of a given classification rule like receiver operating characteristic (ROC) curve [Gronsbell and Cai, 2018], etc. These papers typically propose estimators for a finite dimensional target parameter and study their asymptotic performance, which supplement extensive literature on semi-supervised learning of prediction rules [see Zhu and Goldberg, 2009, for a comprehensive review].…”

“…[2018], Zhang and Bradic [2019]. In this special setting, R is independent with all other variables, so λ * (X) = f * (X)/f * (X | R = 1) = 1 does not need to be estimated, and the estimator δ in Definition 2 reduces to…”