Causal inference in the presence of missing data using a random forest‐based matching algorithm

Abstract

Observational studies require matching across groups over multiple confounding variables. Across the literature, matching algorithms fail to handle this issue. In this way, missing values are regularly imputed prior to being considered in the matching process. However, imputing is not always practical, forcing us to drop an observation due to the deficiency of the chosen algorithm, decreasing the power of the study, and possibly failing to capture crucial latent information. We propose a missing data mechanism to incorporate within an iterative multivariate matching method. The underlying framework utilizes random forest, implemented with surrogate splits as a natural tool in constructing a distance matrix where there might be missing values. The out-put is then easily fed into an optimal matching algorithm. We apply this method to evaluate the effectiveness of Supplemental Instruction (SI) sessions, a voluntary program where students seek additional help, in a large enrollment, bottleneck introductory business statistics course. This is an observational study with two groups, those who attend multiple SI sessions and those who do not, and, as typical in educational data mining, challenged by missing data. Additionally, we perform a data simulation on missingness to further demonstrate the efficacy of our proposed approach.

Facilitated Study Groups for Undergraduate Organic Chemistry: Experience from a Large Public Canadian University

“…Through the use of a wide variation of questions, levels of difficulty, activities, and through consistently seeking student feedback on our methods, we have sought to promote a supportive environment conducive to the learning of a diverse group of students. While we did not assess the impact of FSGs across different demographic groups, previous work has reported that beneficial effects of SI such as the improved likelihood of passing the course are observed in students from different academic and demographic backgrounds (Arendale, 1997;Hillis et al, 2021). In fact, under certain contexts, disadvantaged students such as those from underrepresented minority groups experience preferential benefits from SI relative to their non-disadvantaged counterparts (Yue et al, 2018).…”