Semiparametric Estimation with Data Missing Not at Random Using an Instrumental Variable

Abstract

Missing data occur frequently in empirical studies in health and social sciences, and can compromise our ability to obtain valid inference. An outcome is said to be missing not at random (MNAR) if, conditional on the observed variables, the missing data mechanism still depends on the unobserved outcome. In such settings, identification is generally not possible without imposing additional assumptions. Identification is sometimes possible, however, if an instrumental variable (IV) is observed for all subjects which satisfies the exclusion restriction that the IV affects the missingness process without directly influencing the outcome. In this paper, we provide necessary and sufficient conditions for nonparametric identification of the full data distribution under MNAR with the aid Statistica Sinica: Newly accepted Paper (accepted version subject to English editing) Estimation of MNAR Data with IV 2 of an IV. In addition, we give sufficient identification conditions that are more straightforward to verify in practice. For inference, we focus on estimation of a population outcome mean, for which we develop a suite of semiparametric estimators that extend methods previously developed for data missing at random.Specifically, we propose a novel doubly robust estimator of the mean of an outcome subject to MNAR. For illustration, the methods are used to account for selection bias induced by HIV testing refusal in the evaluation of HIV seroprevalence in Mochudi, Botswana, using interviewer characteristics such as gender, age and years of experience as IVs.

Methodological Context and Communication on Data Quality: Discussion of Bailey (2023)

“…It would be of interest to explore in depth some ways in which the above-mentioned literature may offer additional perspectives on the concerns stated in Bailey (2023), and the related comments in Bailey (2023) Y regarding applications of methodology from Meng (2018), Sun et al (2018), Jackman and Spahn (2019), and Bradley et al (2021).…”

A Self-Censoring Model for Multivariate Nonignorable Nonmonotone Missing Data

“…(2014) and Miao et al. (2016) illustrated that identification is not ensured even for fully parametric models, and fully‐observed auxiliary variables such as shadow variables (D'Haultfœuille, 2010; Miao & Tchetgen Tchetgen, 2016) or instrumental variables (Sun et al., 2018) have been used to achieve identification under nonignorable missingness. However, for multivariate missing data, the missingness of each variable may depend on different partially‐observed variables and the missingness of other variables, which leads to difficulty for specifying provably identifiable models.…”

“…Examples include religious beliefs and sexual preferences in epidemiological studies, smokers not reporting their smoking behavior in insurance applications, and voters not disclosing their political preferences in election surveys. Although this notion of self‐censoring for a univariate outcome has been studied extensively (e.g., D'Haultfœuille, 2010; Wang et al., 2014; Miao et al., 2016; Sun et al., 2018), the multivariate self‐censoring mechanism is rarely studied with only few exceptions. Brown (1990) considered a self‐censoring mechanism for multivariate normal outcomes with a fully‐observed outcome.…”

“…Nonignorable missingness introduces fundamental challenges to identification, where identification means that the full-data distribution or a statistical functional of interest is uniquely determined from the observed-data distribution. For a single missing outcome, Wang et al (2014) and illustrated that identification is not ensured even for fully parametric models, and fully-observed auxiliary variables such as shadow variables (D'Haultfoeuille, 2010;Miao & Tchetgen Tchetgen, 2016) or instrumental variables (Sun et al, 2018) have been used to achieve identification under nonignorable missingness. However, for multivariate missing data, the missingness of each variable may depend on different partially-observed variables and the missingness of other variables, which leads to difficulty for specifying provably identifiable models.…”

Testing the missing at random assumption in generalized linear models in the presence of instrumental variables

“…Wang et al (2019) recently considered propensity selection and data modeling with some penalized information criteria. Sun et al (2019) studied identifiability in semiparametric estimation with instrumental variables; see also Miao and Tchetgen (2018).…”