Deep Learning-based Propensity Scores for Confounding Control in Comparative Effectiveness Research

Abstract

Background: Due to the non-randomized nature of real-world data, prognostic factors need to be balanced, which is often done by propensity scores (PSs). This study aimed to investigate whether autoencoders, which are unsupervised deep learning architectures, might be leveraged to compute PS. Methods:We selected patient-level data of 128,368 first-line treated cancer patients from the Flatiron Health EHR-derived de-identified database. We trained an autoencoder architecture to learn a lowerdimensional patient representation, which we used to compute PS. To compare the performance of an autoencoder-based PS with established methods, we performed a simulation study. We assessed the balancing and adjustment performance using standardized mean differences, root mean square errors (RMSE), percent bias, and confidence interval coverage. To illustrate the application of the autoencoder-based PS, we emulated the PRONOUNCE trial by applying the trial's protocol elements within an observational database setting, comparing two chemotherapy regimens.Results: All methods but the manual variable selection approach led to well-balanced cohorts with average standardized mean differences <0.1. LASSO yielded on average the lowest deviation of resulting estimates (RMSE 0.0205) followed by the autoencoder approach (RMSE 0.0248). Altering the hyperparameter setup in sensitivity analysis, the autoencoder approach led to similar results as LASSO (RMSE 0.0203 and 0.0205, respectively). In the case study, all methods provided a similar conclusion with point estimates clustered around the null (e.g., HR autoencoder 1.01 [95% confidence interval = 0.80, 1.27] vs. HR PRONOUNCE 1.07 [0.83, 1.36]). Conclusions: Autoencoder-based PS computation was a feasible approach to control for confounding but did not perform better than some established approaches like LASSO.

High-Throughput Clinical Trial Emulation with Real World Data and Machine Learning: A Case Study of Drug Repurposing for Alzheimer’s Disease

“…Lots of recent research efforts have been devoted to developing complex deep learning-based models for propensity score based modeling 7,[40][41][42][43] In this paper, after emulating hundreds of thousands of trials from two large-scale RWD warehouses, we found that one LSTM-PS 7 , which is a representative deep learning based PS method, did not outperform LR-PS. Our study also highlighted the importance of model selection and we proposed our own strategy under which we demonstrated LR-PS outperformed gradient boosting tree-based PS models and deep multi-layer peceptron-based PS models as well in terms of balancing performance and the number of generated repurposing hypotheses.…”

Machine learning for improving high‐dimensional proxy confounder adjustment in healthcare database studies: An overview of the current literature

“…26 The use of neural networks for extracting confounder information by modeling complex coding patterns is promising but examples are limited. 27,28 2.1 | Challenges in generating features for proxy adjustment from electronic health records An important limitation of current high-dimensional proxy confounder adjustment approaches is that they can only use structured electronic healthcare information. However, much of the essential confounder information, such as patient-reported symptoms, severity, stage and prognosis of the disease, and functional status, is frequently recorded in free-text notes or reports in electronic health records (EHRs) that are substantially underutilized for confounding adjustment.…”

“…For example, recent work has proposed using neural networks to model a patient's full course of care to consider temporal sequences of a specific course of treatment 26 . The use of neural networks for extracting confounder information by modeling complex coding patterns is promising but examples are limited 27,28 …”

Generalising uncertainty improves accuracy and safety of deep learning analytics applied to oncology

“…CT scans [6], histopathology slides [7], genomics [8], transcriptomics [9], [10], and most recently, integrated approaches with various data types [11], [12]. In general, studies using DL show excellent predictive performance, providing hope for successful translation into clinical practice [13], [14]. However, prediction accuracy in DL comes with potential pitfalls which need to be overcome before wider adoption can be eventuated [15].…”