Validation of multisource electronic health record data: an application to blood transfusion data

Abstract

BackgroundAlthough data from electronic health records (EHR) are often used for research purposes, systematic validation of these data prior to their use is not standard practice. Existing validation frameworks discuss validity concepts without translating these into practical implementation steps or addressing the potential influence of linking multiple sources. Therefore we developed a practical approach for validating routinely collected data from multiple sources and to apply it to a blood transfusion data warehouse to evaluate the usability in practice.MethodsThe approach consists of identifying existing validation frameworks for EHR data or linked data, selecting validity concepts from these frameworks and establishing quantifiable validity outcomes for each concept. The approach distinguishes external validation concepts (e.g. concordance with external reports, previous literature and expert feedback) and internal consistency concepts which use expected associations within the dataset itself (e.g. completeness, uniformity and plausibility). In an example case, the selected concepts were applied to a transfusion dataset and specified in more detail.ResultsApplication of the approach to a transfusion dataset resulted in a structured overview of data validity aspects. This allowed improvement of these aspects through further processing of the data and in some cases adjustment of the data extraction. For example, the proportion of transfused products that could not be linked to the corresponding issued products initially was 2.2% but could be improved by adjusting data extraction criteria to 0.17%.ConclusionsThis stepwise approach for validating linked multisource data provides a basis for evaluating data quality and enhancing interpretation. When the process of data validation is adopted more broadly, this contributes to increased transparency and greater reliability of research based on routinely collected electronic health records.Electronic supplementary materialThe online version of this article (doi:10.1186/s12911-017-0504-7) contains supplementary material, which is available to authorized users.

Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience

“…[7,9,10,26,27] However, less has been published about how to operationalize these approaches, particularly in multisite clinical datasets. [7,28,29] Kahn and colleagues describe a conceptual model and a number of computational rules to explore data quality in electronic health record-based research. [7] Similarly, van Hoeven and colleagues articulate an approach to assessing the validity of linked data using computational methods and report its application in a specific case using transfusion data.…”

“…[7] Similarly, van Hoeven and colleagues articulate an approach to assessing the validity of linked data using computational methods and report its application in a specific case using transfusion data. [28] Terry and colleagues developed 11 measures of quality for primary health care data extracted from electronic medical records. [29] Each of these studies admirably documents the process of operationalizing conceptual data quality frameworks into real-world applications.…”

How to validate a diagnosis recorded in electronic health records

“…Measuring data validity is therefore needed to establish whether the values “make sense” [5, 6]. When considering answering a research question using EHR, a researcher should always contemplate the following question: are we measuring what we are intending to measure?…”

“…The remainder of this section outlines a non-exhaustive list of eight common techniques to validate diagnostic algorithms with references to examples, ranked loosely from most to least resource-intensive. There are other possible techniques, including studying the completeness, plausibility, uniformity and time patterns of the data, which are not described in detail with examples in this study [5]. Not all these validation techniques are necessarily implementable in each database and some techniques are resource-intensive, while others provide only an indication of the validity.…”

Assessing the quality of clinical and administrative data extracted from hospitals: The General Medicine Inpatient Initiative (GEMINI) experience

“…[7,9,10,26,27] However, less has been published about how to operationalize these approaches, particularly in multi-site clinical datasets. [7,28,29] Kahn and colleagues describe a conceptual model and a number of computational rules to explore data quality in electronic health record-based research. [7] Similarly, van Hoeven and colleagues articulate an approach to assessing the validity of linked data using computational methods and report its application in a specific case using transfusion data.…”