Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review

Abstract

ObjectivesRecord review is the most used method to quantify patient safety. We systematically reviewed the reliability and validity of adverse event detection with record review.DesignA systematic review of the literature.MethodsWe searched PubMed, EMBASE, CINAHL, PsycINFO and the Cochrane Library and from their inception through February 2015. We included all studies that aimed to describe the reliability and/or validity of record review. Two reviewers conducted data extraction. We pooled κ values (κ) and analysed the differences in subgroups according to number of reviewers, reviewer experience and training level, adjusted for the prevalence of adverse events.ResultsIn 25 studies, the psychometric data of the Global Trigger Tool (GTT) and the Harvard Medical Practice Study (HMPS) were reported and 24 studies were included for statistical pooling. The inter-rater reliability of the GTT and HMPS showed a pooled κ of 0.65 and 0.55, respectively. The inter-rater agreement was statistically significantly higher when the group of reviewers within a study consisted of a maximum five reviewers. We found no studies reporting on the validity of the GTT and HMPS.ConclusionsThe reliability of record review is moderate to substantial and improved when a small group of reviewers carried out record review. The validity of the record review method has never been evaluated, while clinical data registries, autopsy or direct observations of patient care are potential reference methods that can be used to test concurrent validity.

Adverse events in hospitalised cancer patients: a comparison to a general hospital population

“…The reliability of record reviewing is moderate to sustainable when done by a small group of reviewers [9]. In our study, seven different teams (21 reviewers) did the review.…”

“…The Institute for Healthcare Improvement's Global Trigger Tool (IHI GTT) is widely used as a method to measure and monitor AEs in general hospitalised patients [3,6]. Despite this method's high sensitivity and specificity in detecting iatrogenic harm, there are limitations [7][8][9]. One Danish study raises methodological concerns of the IHI GTT, not being specific enough in monitoring harm in cancer patients [10].…”

A study of cases reported as incidents in a public hospital from 2011 to 2014

“…Furthermore, the occurrence of AE constitutes an important factor for evaluating patient safety, which can be understood as reducing the associated risk of unnecessary damage to health care down to a minimum acceptable level (6) . Several current studies report a strong link between safety and quality in health (9,11) . In this perspective, safety is one of the dimensions of health service quality, being an inseparable attribute (9) .…”

“…Adverse events (AE) have an important impact on the Unified Health System (Sistema Único de Saúde -SUS) which result in increased morbidity, mortality, patient treatment time and care costs, and are reflected in other fields of social and economic life in all countries (8)(9)(10) . Notifying and acknowledging the existence of AE facilitate investigation of care quality, because failures do happen and are important for evaluating health services (4,7,(11)(12)(13) . Furthermore, the occurrence of AE constitutes an important factor for evaluating patient safety, which can be understood as reducing the associated risk of unnecessary damage to health care down to a minimum acceptable level (6) .…”

“…In this perspective, safety is one of the dimensions of health service quality, being an inseparable attribute (9) . Thus, health systems that reduce the risk of injury to the patient to a minimum are undoubtedly increasing the quality of their services (5,9,11) . The aim of this study was to analyze the incidents reported in a public hospital in the Federal District, the characteristics of patients involved, and the outcomes for the patient in the period of 2011 to 2014.…”