Assessing the quality of clinical and administrative data extracted from hospitals: The General Medicine Inpatient Initiative (GEMINI) experience

Abstract

Objective: Large clinical databases are increasingly being used for research and quality improvement, but there remains uncertainty about how computational and manual approaches can be used together to assess and improve the quality of extracted data. The General Medicine Inpatient Initiative (GEMINI) database extracts and standardizes a broad range of data from clinical and administrative hospital data systems, including information about attending physicians, room transfers, laboratory tests, diagnostic imaging reports, and outcomes such as death in-hospital. We describe computational data quality assessment and manual data validation techniques that were used for GEMINI. Methods: The GEMINI database currently contains 245,559 General Internal Medicine patient admissions at 7 hospital sites in Ontario, Canada from 2010-2017. We performed 7 computational data quality checks followed by manual validation of 23,419 selected data points on a sample of 7,488 patients across participating hospitals. After iteratively re-extracting data as needed based on the computational data quality checks, we manually validated GEMINI data against the data that could be obtained using the hospital electronic medical record (i.e. the data clinicians would see when providing care), which we considered the gold standard. We calculated accuracy, sensitivity, specificity, and positive and negative predictive values of GEMINI data. Results: Computational checks identified multiple data quality issues: for example, the inclusion of cancelled radiology tests, a time shift of transfusion data, and mistakenly processing the symbol for sodium, Na, as a missing value. Manual data validation revealed that GEMINI data were ultimately highly reliable compared to the gold standard across nearly all data tables. One important data quality issue was identified by manual validation that was not detected by computational checks, which was that the dates and times of blood transfusion data at one site were not reliable. This resulted in low sensitivity (66%) and positive predictive value (75%) for blood transfusion data at that site. Apart from this single issue, GEMINI data were highly reliable across all data tables, with high overall accuracy (ranging from 98-100%), sensitivity (95-100%), specificity (99-100%), positive predictive value (93-100%), and negative predictive value (99-100%) compared to the gold standard. Discussion and Conclusion: Iterative assessment and improvement of data quality based primarily on computational checks permitted highly reliable extraction of multisite clinical and administrative data. Computational checks identified nearly all of the data quality issues in this initiative but one critical quality issue was only identified during manual validation. Combining computational checks and manual validation may be the optimal method for assessing and improving the quality of large multi-site clinical databases.

Modelling resource requirements and physician staffing to provide virtual urgent medical care for residents of long-term care homes: a cross-sectional study

“…GEMINI data go through a statistical and manual screening process that includes range checks and missing data checks during collection and harmonization. 9 This process is iterative, and data are re-extracted when problems are identified. No additional data-cleaning steps were taken in the preparation of the data for this study.…”

“…All data were linked using unique identifiers and a subset of the data demonstrated 98%-100% accuracy compared with chart review. 9 We identified all patients who were admitted from LTC facilities using the institution number developed by the Ministry of Health and Long-Term Care Master Numbering System in our administrative data. 10 These facilities included nursing homes and facilities offering complex continuing care but not rehabilitation hospitals.…”

Bedspacing and clinical outcomes in general internal medicine: A retrospective, multicenter cohort study

“…Patient‐level data were collected from electronic and administrative sources at each participating hospital, as described previously 5 . GEMINI data have been rigorously validated against manual review of medical records and found to be highly reliable 6 . Patient demographics, diagnoses, and discharge disposition were collected from hospital administrative sources, as reported to the Canadian Institute for Health Information.…”

Association of diabetes with frequency and cost of hospital admissions: a retrospective cohort study

“…Unique identifiers were used to link all data, and analyses on a subset of data showed 98%-100% accuracy compared with chart review. 12 Patients with diabetes were identified using CIHI Diagnosis Code (Group 10, Field 02) which conveys International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10) codes as recorded in hospital administrative databases.…”