Sensitivity and specificity of an algorithm based on medico-administrative data to identify hospitalized patients with major bleeding presenting to an emergency department

Abstract

Background Validation studies on an ICD-10-based algorithm to identify major bleeding events are scarce, and mostly focused on positive predictive values. Objective To evaluate the sensitivity and specificity of an ICD-10-based algorithm in adult patients referred to hospital. Methods This was a cross-sectional, retrospective analysis. Among all hospital stays of adult patients referred to Rennes University Hospital, France, through the emergency ward in 2014, we identified major bleeding events according to an index test based on a list of ICD-10 diagnoses. As a reference, a two-step process was applied: firstly, a computerized request for electronic health records from the emergency ward, using several hemorrhage-related diagnostic codes and specific emergency therapies so as to discard stays with a very low probability of bleeding; secondly, a chart review of selected records was conducted by a medical expert blinded to the index test results and each hospital stay was classified into one of two exclusive categories: major bleeding or no major bleeding, according to pre-specified criteria. Results Out of 16,012 hospital stays, the reference identified 736 major bleeding events and left 15,276 stays considered as without the target condition. The index test identified 637 bleeding events: 293 intracranial hemorrhages, 197 gastrointestinal hemorrhages and 147 other bleeding events. Overall, sensitivity was 65% (95%CI, 62 to 69), and specificity was 99.0%. We observed differential sensitivity and specificity across bleeding types, with the highest values for intracranial hemorrhage. Positive predictive values ranged from 59% for “other” bleeding events, to 71% (95%CI, 65 to 78) for gastrointestinal hemorrhage, and 96% for intracranial hemorrhage. Conclusions Low sensitivity and differential measures of accuracy across bleeding types support the need for specific data collection and medical validation rather than using an ICD-10-based algorithm for assessing the incidence of major bleeding.

Rivaroxaban was found to be noninferior to warfarin in routine clinical care: A retrospective noninferiority cohort replication study

“…Table S1 shows the ICD‐10 (International Statistical Classification of Diseases and Related Health Problems 10th Revision) codes that were utilized to identify the primary and secondary outcomes. Diagnostic accuracy of major bleeding and death ICD‐10 algorithms were assessed in a couple of validation studies with mixed results 27,28 . Studies that assessed the validity of ICD‐10 algorithms of other outcomes were not identified in the published literature.…”

“…Diagnostic accuracy of major bleeding and death ICD-10 algorithms were assessed in a couple of validation studies with mixed results. 27,28 Studies that assessed the validity of ICD-10 algorithms of other outcomes were not identified in the published literature.…”

Major bleeding risk and mortality associated with antiplatelet drugs in real-world clinical practice. A prospective cohort study

“…This observation was confirmed in the present study for all antiplatelet drugs, in monotherapy or dual therapy, but with overall lower incidence rates. This could be explained by the strict definition of the severity of major bleeding used in our study and by the systematic medical validation of all events [23].…”

“…All bleeding events were medically validated using pre-specified criteria which were somewhat consistent with the "severe or life-threatening' item from GUSTO. The systematic medical validation of all bleeding limited the overestimation of incidence rates of major bleeding [23]. Exposure in our study was based on reimbursement claims data.…”

Major bleeding risk associated with oral anticoagulant in real clinical practice. A multicentre 3‐year period population‐based prospective cohort study

“…A pilot study found good sensitivity for the computer search. 33 In each area, a referent expert medical doctor checked all records identified for oral anticoagulant exposure and severe bleeding criteria (see below for details).…”