Generating and evaluating a propensity model using textual features from electronic medical records

Abstract

Background Propensity score (PS) methods are commonly used to control for confounding in comparative effectiveness studies. Electronic health records (EHRs) contain much unstructured data that could be used as proxies for potential confounding factors. The goal of this study was to assess whether the unstructured information can also be used to construct PS models that would allow to properly deal with confounding. We used an example of coxibs (Cox-2 inhibitors) vs. traditional NSAIDs and the risk of upper gastro-intestinal bleeding as example, since this association is often confounded due to channeling of coxibs to patients at higher risk of upper gastro-intestinal bleeding. Methods In a cohort study of new users of nonsteroidal anti-inflammatory drugs (NSAIDs) from the Dutch Integrated Primary Care Information (IPCI) database, we identified all patients who experienced an upper gastrointestinal bleeding (UGIB). We used a large-scale regularized regression to fit two PS models using all structured and unstructured information in the EHR. We calculated hazard ratios (HRs) to estimate the risk of UGIB among selective cyclo-oxygenase-2 (COX-2) inhibitor users compared to nonselective NSAID (nsNSAID) users. Results The crude hazard ratio of UGIB for COX-2 inhibitors compared to nsNSAIDs was 0.50 (95% confidence interval 0.18–1.36). Matching only on age resulted in an HR of 0.36 (0.11–1.16), and of 0.35 (0.11–1.11) when further adjusted for sex. Matching on PS only, the first model yielded an HR of 0.42 (0.13–1.38), which reduced to 0.35 (0.96–1.25) when adjusted for age and sex. The second model resulted in an HR of 0.42 (0.13–1.39), which dropped to 0.31 (0.09–1.08) after adjustment for age and sex. Conclusions PS models can be created using unstructured information in EHRs. An incremental benefit was observed by matching on PS over traditional matching and adjustment for covariates.

Scalable Feature Engineering from Electronic Free Text Notes to Supplement Confounding Adjustment of Claims‐Based Pharmacoepidemiologic Studies

“…Studies have also shown that models that optimize prediction can underfit for purposes of confounding adjustment and are not necessarily optimal for reducing bias in estimated treatment effects 43 . Recent work independent of ours has explored the use of large‐scale feature engineering from unstructured EHR text for improved confounding control for propensity score analyses 46 . This work found benefits confounding control, but the benefits were incremental and findings were limited to a single empirical study 46 .…”

“…Recent work independent of ours has explored the use of large‐scale feature engineering from unstructured EHR text for improved confounding control for propensity score analyses 46 . This work found benefits confounding control, but the benefits were incremental and findings were limited to a single empirical study 46 . Determining to what extent the addition of these EHR‐generated features can improve confounding control is a more difficult problem that we leave to future research.…”

“…46 This work found benefits confounding control, but the benefits were incremental and findings were limited to a single empirical study. 46 Determining to what extent the addition of these EHR-generated features can improve confounding control is a more difficult problem that we leave to future research.…”

Machine learning for improving high‐dimensional proxy confounder adjustment in healthcare database studies: An overview of the current literature

“…Natural language processing (NLP) is a subfield of machine learning that can be used to generate variables from unstructured free text 31 . NLP methods are increasingly used to identify health outcomes from EHRs, but the application of NLP algorithms for purposes of identifying high‐dimensional sets of confounding factors is limited 32 . More research is needed on the use of NLP algorithms for generating high‐dimensional sets of proxy confounders and the value of unstructured EHR data in proxy adjustment.…”