Prediction Accuracy With Electronic Medical Records Versus Administrative Claims

Abstract

Objective: The objective of this study was to evaluate the incremental predictive power of electronic medical record (EMR) data, relative to the information available in more easily accessible and standardized insurance claims data. Data and Methods: Using both EMR and Claims data, we predicted outcomes for 118,510 patients with 144,966 hospitalizations in 8 hospitals, using widely used prediction models. We use cross-validation to prevent overfitting and tested predictive performance on separate data that were not used for model training. Main Outcomes: We predict 4 binary outcomes: length of stay (≥7 d), death during the index admission, 30-day readmission, and 1-year mortality. Results: We achieve nearly the same prediction accuracy using both EMR and claims data relative to using claims data alone in predicting 30-day readmissions [area under the receiver operating characteristic curve (AUC): 0.698 vs. 0.711; positive predictive value (PPV) at top 10% of predicted risk: 37.2% vs. 35.7%], and 1-year mortality (AUC: 0.902 vs. 0.912; PPV: 64.6% vs. 57.6%). EMR data, especially from the first 2 days of the index admission, substantially improved prediction of length of stay (AUC: 0.786 vs. 0.837; PPV: 58.9% vs. 55.5%) and inpatient mortality (AUC: 0.897 vs. 0.950; PPV: 24.3% vs. 14.0%). Results were similar for sensitivity, specificity, and negative predictive value across alternative cutoffs and for using alternative types of predictive models. Conclusion: EMR data are useful in predicting short-term outcomes. However, their incremental value for predicting longer-term outcomes is smaller. Therefore, for interventions that are based on long-term predictions, using more broadly available claims data is equally effective.

Predictive modeling of U.S. health care spending in late life

“…A natural question is whether these results could change if we had better predictions, for example using higher quality data such as electronic medical records. The available evidence, while limited, suggests that relative to using only (detailed) claims data, the incremental predictive power obtained from electronic medical records (15, 16) or subjective physician predictions (17, 18) is relatively small. Moreover, such data are arguably less relevant for national policy, which needs to be based on standardized, nationally available data.…”

Comparing the Predictive Effects of Patient Medication Adherence Indices in Electronic Health Record and Claims-Based Risk Stratification Models

“…EHR data are increasingly being used, along with insurance claims, to improve population stratification efforts. 11,24,[28][29][30][31][32] The research team developed risk stratification models based on EHR and claims data of 70,054 patients to examine the added values of various medication adherence indices in predicting hospitalization and annual health care total cost (ie, the same year or the following year). Adherence measures of MPR did not boost model performances considerably, with PFR adding minimal improvements.…”

Development of Deep Learning Models for Predicting in-Hospital Mortality using an Administrative Claims Database

“…In a previous study, the predictive abilities of models with administrative claims data alone were compared with those of models with electronic medical records combined with administrative claims data [22]. The predictive abilities of the models with electronic medical records were higher because the electronic medical records included sophisticated information on disease-speci c severity.…”