Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience

Abstract

Objective Large clinical databases are increasingly used for research and quality improvement. We describe an approach to data quality assessment from the General Medicine Inpatient Initiative (GEMINI), which collects and standardizes administrative and clinical data from hospitals. Methods The GEMINI database contained 245 559 patient admissions at 7 hospitals in Ontario, Canada from 2010 to 2017. We performed 7 computational data quality checks and iteratively re-extracted data from hospitals to correct problems. Thereafter, GEMINI data were compared to data that were manually abstracted from the hospital’s electronic medical record for 23 419 selected data points on a sample of 7488 patients. Results Computational checks flagged 103 potential data quality issues, which were either corrected or documented to inform future analysis. For example, we identified the inclusion of canceled radiology tests, a time shift of transfusion data, and mistakenly processing the chemical symbol for sodium (“Na”) as a missing value. Manual validation identified 1 important data quality issue that was not detected by computational checks: transfusion dates and times at 1 site were unreliable. Apart from that single issue, across all data tables, GEMINI data had high overall accuracy (ranging from 98%–100%), sensitivity (95%–100%), specificity (99%–100%), positive predictive value (93%–100%), and negative predictive value (99%–100%) compared to the gold standard. Discussion and Conclusion Computational data quality checks with iterative re-extraction facilitated reliable data collection from hospitals but missed 1 critical quality issue. Combining computational and manual approaches may be optimal for assessing the quality of large multisite clinical databases.

Characteristics and outcomes of hospital admissions for COVID-19 and influenza in the Toronto area

“…We collected administrative and clinical data from hospital information systems for GEMINI, as previously described. 12,13 We collected patient demographics, hospital resource use and outcomes from hospitals, as reported to the Canadian Institute for Health Information (CIHI) Discharge Abstract Database and the National Ambulatory Care Reporting System. We extracted additional clinical data, including laboratory test results, radiology tests, vital signs and inhospital medication orders from hospital information systems (see Appendix 1, available at www.cmaj.ca/lookup/doi/10.1503/ cmaj.202795/tab-related-content, for details regarding data quality).…”

Machine Learning Analysis for Data Incompleteness (MADI): Analyzing the Data Completeness of Patient Records Using a Random Variable Approach to Predict the Incompleteness of Electronic Health Records

“…[5] • Completeness Scoring Variable (CSV): Where, CSVz is the completeness measure of column z and its value is between 0 and 1: According to Algorithm 1, the investigators compute DIM for each column of the experimental dataset. This is followed by the generation of the histogram of the entire dataset [6], based on the incompleteness ratio of each column [7].…”

Patient characteristics, clinical care, resource use, and outcomes associated with hospitalization for COVID-19 in the Toronto area

“…Administrative and clinical data were collected from hospital information systems for GEMINI as has previously been described in detail 19,20 . Manual validation of a subset of GEMINI data found 98-100% accuracy compared to detailed medical record review.…”

“…Manual validation of a subset of GEMINI data found 98-100% accuracy compared to detailed medical record review. 20 Patient demographics, hospital resource use, and outcomes were collected from hospitals as reported to the Canadian Institute for Health Information (CIHI) Discharge Abstract Database and National Ambulatory Care Reporting System. Additional clinical data, including laboratory test results, radiology tests, vital signs and in-hospital medication orders were extracted from hospital information systems.…”