Measuring and controlling medical record abstraction (MRA) error rates in an observational study

Abstract

Background Studies have shown that data collection by medical record abstraction (MRA) is a significant source of error in clinical research studies relying on secondary use data. Yet, the quality of data collected using MRA is seldom assessed. We employed a novel, theory-based framework for data quality assurance and quality control of MRA. The objective of this work is to determine the potential impact of formalized MRA training and continuous quality control (QC) processes on data quality over time. Methods We conducted a retrospective analysis of QC data collected during a cross-sectional medical record review of mother-infant dyads with Neonatal Opioid Withdrawal Syndrome. A confidence interval approach was used to calculate crude (Wald’s method) and adjusted (generalized estimating equation) error rates over time. We calculated error rates using the number of errors divided by total fields (“all-field” error rate) and populated fields (“populated-field” error rate) as the denominators, to provide both an optimistic and a conservative measurement, respectively. Results On average, the ACT NOW CE Study maintained an error rate between 1% (optimistic) and 3% (conservative). Additionally, we observed a decrease of 0.51 percentage points with each additional QC Event conducted. Conclusions Formalized MRA training and continuous QC resulted in lower error rates than have been found in previous literature and a decrease in error rates over time. This study newly demonstrates the importance of continuous process controls for MRA within the context of a multi-site clinical research study.

Comparing Medical Record Abstraction (MRA) Error Rates in an Observational Study to Pooled Rates Identified in the Data Quality Literature

“…Approximately 1,800 cases were abstracted across all study sites, of which a subset of cases (over 200) underwent a formalized QC process to identify data quality errors. Additional information on the ACT NOW CE Study, 14 including details on the MRA training 32 and QC process, 13 has been published elsewhere.…”

“…12,37 The ACT NOW CE Study was unique in that it implemented and evaluated formalized MRA training and continuous QC processes in an effort to improve data quality. 13,32 To our knowledge, this was the rst time that an MRA-QC framework, such as that published by Zozus and colleagues, 12,13,32 was implemented and evaluated throughout the course of an ongoing clinical research study. There is a lack of evidence in the literature to suggest that previous clinical studies had implemented any formalized training or QC process to address error rates.…”

“…12 In an effort to mitigate the inherent risks associated with MRA, we developed and employed a theorybased, quality control (QC) framework to support MRA activities in clinical research. 13 Our MRA-QC framework involves standardized MRA training prior to study implementation, as well as a continuous QC process carried out throughout the course of the study. We implemented and evaluated the MRA-QC framework within the context of the Advancing Clinical Trials in Neonatal Opioid Withdrawal Current Experience (ACT NOW CE) Study 14 to measure the in uence of formalized MRA training and continuous QC on data quality.…”

A hybrid approach to scalable real-world data curation by machine learning and human experts

“…6 Errors caused by oversight (e.g., missing a biomarker test result) or miscoding (e.g., mistaking a second primary tumor for a distant metastasis) have the potential to bias downstream analyses. 7…”

“…6 Errors caused by oversight (e.g., missing a biomarker test result) or miscoding (e.g., mistaking a second primary tumor for a distant metastasis) have the potential to bias downstream analyses. 7 To produce datasets that are fit for use in retrospective analyses, the RWD curation process must be reliable and unbiased. Traditionally, this has been achieved through manual curation by highly-trained abstractors.…”