Use of Recommended Real-World Methods for Electronic Health Record Data Analysis Has Not Improved Over 10 Years

Abstract

Background and PurposeTo document the use of recommended Real-World Methods (RWM) in Electronic Health Record (EHR)-based analysis in biomedical research over 10 years.MethodsSampled-article scoping review of methods used in EHR-based biomedical research. We developed a search strategy to identify reports of biomedical research based on EHR data and systematically sampled articles from different ranges of years (epochs) between 2010 and 2019 to establish a trajectory of use of recommended RWM. Methods were classified by 3 phases of research: pre-analytic (missing data), analytic (specific methods), and post-analytic (sensitivity analysis). The primary outcome was the proportion of studies using recommended RWM within each epoch. Meta-regressions were performed to examine trends.Data SynthesisFive epochs were defined between 2010 and 2019 with 35 studies selected per epoch as pre-defined by a sample size calculation. Of the 175 articles reviewed, 70 (40.%) reported recommended RWM in any of the 3 phases of research. The breakdown for the most recent year in the dataset, 2019, was 14.% (95% confidence interval 2.7%, 26.%), 14.% (2.7%, 26.%), and 11.% (0.89%, 22.%), for assessing missing data, using specific methods, and performing sensitivity analysis, respectively. Only 3.4 % of studies used appropriate methods for each phase of research. Meta-regression slopes for each of the three phases were statistically 0.Limitation and ConclusionsThe underuse of recommended Real-World Methods (RWM) in EHR-based biomedical research remains a concern, with less than 50% of reports using these methods in any phase of research over the last decade. This lack of use indicates a continued risk of bias in the EHR-based literature.