Efficient Discovery of Heterogeneous Treatment Effects in Randomized Experiments via Anomalous Pattern Detection

McFowland, Edward; Somanchi, Sriram; Neill, Daniel B.

doi:10.2139/ssrn.3155352

Cited by 11 publications

(16 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application of our approach in empirical contexts has the potential to improve the precision and robustness of estimations, and thus subsequent decision-making. Meanwhile, our approach shows the possibility of automatically generating candidate instruments based on an ensemble learning technique, which complements the emerging literature on the use of machine learning methods for causal inference (e.g., Athey and Imbens, 2016;McFowland III et al, 2018). At the core of ForestIV is the fundamental trade-off between an estimator's bias and variance, which together describe its statistical risk.…”

Section: Discussionmentioning

confidence: 81%

Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem

et al. 2019

Self Cite

View full text Add to dashboard Cite

Combining machine learning with econometric analysis is becoming increasingly prevalent in both research and practice. A common empirical strategy involves the application of predictive modeling techniques to "mine" variables of interest from available data, followed by the inclusion of those variables into an econometric framework, with the objective of estimating causal effects. Recent work highlights that, because the predictions from machine learning models are inevitably imperfect, econometric analyses based on the predicted variables are likely to suffer from bias due to measurement error. We propose a novel approach to mitigate these biases, leveraging the ensemble learning technique known as the random forest. We propose employing random forest not just for prediction, but also for generating instrumental variables to address the measurement error embedded in the prediction. The random forest algorithm performs best when comprised of a set of trees that are individually accurate in their predictions, yet which also make "different" mistakes, i.e., have weakly correlated prediction errors. A key observation is that these properties are closely related to the relevance and exclusion requirements of valid instrumental variables. We design a data-driven procedure to select tuples of individual trees from a random forest, in which one tree serves as the endogenous covariate and the other trees serve as its instruments. Simulation experiments demonstrate the efficacy of the proposed approach in mitigating estimation biases, and its superior performance over three alternative methods for bias correction.

show abstract

Section: Discussionmentioning

confidence: 81%

Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Researchers have previously investigated the heterogeneous treatment effects for binary outcomes [1] and interventions [5].…”

Section: Discussionmentioning

confidence: 99%

“…Researchers have previously investigated the heterogeneous treatment effects for binary outcomes[1] and interventions[5]. To the best of our knowledge, this work is one of the first to specifically investigate how complex heterogeneous treatment effects of sequential interventions could be studied in a simplified manner.…”

Section: Discussionmentioning

confidence: 99%

Hierarchical Representation of Complex Intervention Sequences for Automated Subgroup Analysis in Critical Care Settings

Trivedi

Ogallo²,

Tadesse³

2021

Preprint

View full text Add to dashboard Cite

Our understanding of the impact of interventions in critical care is limited by the lack of techniques that represent and analyze complex intervention spaces applied across heterogeneous patient populations. Existing work has mainly focused on selecting a few interventions and representing them as binary variables, resulting in oversimplification of intervention representation. The goal of this study is to find effective representations of sequential interventions to support intervention effect analysis. To this end, we have developed Hi-RISE (Hierarchical Representation of Intervention Sequences), an approach that transforms and clusters sequential interventions into a latent space, with the resulting clusters used for heterogenous treatment effect analysis. We apply this approach to the MIMIC III dataset and identified intervention clusters and corresponding subpopulations with peculiar odds of 28-day mortality. Our approach may lead to a better understanding of the subgroup-level effects of sequential interventions and improve targeted intervention planning in critical care settings.

show abstract

“…Non-parametric Scan Statistics (NPSS) Group-based subset scanning uses NPSS that has been used in other pattern detection methods (McFowland III et al, 2013;McFowland et al, 2018;Chen & Neill, 2014;Cintas et al, 2020;Akinwande et al, 2020). Given that NPSS makes minimal assumptions on the underlying distribution of node activations, our approach has the ability to scan across different type of layers and activation functions.…”

Section: Proposed Approach: Group-based Subset Scanning Over the Crea...mentioning

confidence: 99%

Towards creativity characterization of generative models via group-based subset scanning

Cintas¹,

Das²,

Quanz³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep generative models, such as Variational Autoencoders (VAEs), have been employed widely in computational creativity research. However, such models discourage out-of-distribution generation to avoid spurious sample generation, limiting their creativity. Thus, incorporating research on human creativity into generative deep learning techniques presents an opportunity to make their outputs more compelling and human-like. As we see the emergence of generative models directed to creativity research, a need for machine learning-based surrogate metrics to characterize creative output from these models is imperative. We propose group-based subset scanning to quantify, detect, and characterize creative processes by detecting a subset of anomalous node-activations in the hidden layers of generative models. Our experiments on original, typically decoded, and "creatively decoded" (Das et al., 2020) image datasets reveal that the proposed subset scores distribution is more useful for detecting creative processes in the activation space rather than the pixel space. Further, we found that creative samples generate larger subsets of anomalies than normal or non-creative samples across datasets. The node activations highlighted during the creative decoding process are different from those responsible for normal sample generation.

show abstract

Efficient Discovery of Heterogeneous Treatment Effects in Randomized Experiments via Anomalous Pattern Detection

Cited by 11 publications

References 42 publications

Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem

Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem

Hierarchical Representation of Complex Intervention Sequences for Automated Subgroup Analysis in Critical Care Settings

Towards creativity characterization of generative models via group-based subset scanning

Contact Info

Product

Resources

About