Evaluation of an Artificial Intelligence Model for Identification of Intracranial Hemorrhage Subtypes on Computed Tomography of the Head

Abstract

BACKGROUND Intracranial hemorrhage is a critical finding on computed tomography (CT) of the head. This study compared the accuracy of an artificial intelligence (AI) model (Annalise Enterprise CTB Triage Trauma) to consensus neuroradiologist interpretations in detecting 4 hemorrhage subtypes: acute subdural/epidural hematoma, acute subarachnoid hemorrhage, intra‐axial hemorrhage, and intraventricular hemorrhage. METHODS A retrospective stand‐alone performance assessment was conducted on data sets of cases of noncontrast CT of the head acquired between 2016 and 2022 at 5 hospitals in the United States for each hemorrhage subtype. The cases were obtained from patients aged ≥18 years. The positive cases were selected on the basis of the original clinical reports using natural language processing and manual confirmation. The negative cases were selected by taking the next negative case acquired from the same CT scanner after positive cases. Each case was interpreted independently by up to 3 neuroradiologists to establish consensus interpretations. Each case was then interpreted by the AI model for the presence of the relevant hemorrhage subtype. The neuroradiologists were provided with the entire CT study. The AI model separately received thin (≤1.5 mm) and thick (>1.5 and ≤5 mm) axial series as available. RESULTS The 4 cohorts included 571 cases of acute subdural/epidural hematoma, 310 cases of acute subarachnoid hemorrhage, 926 cases of intra‐axial hemorrhage, and 199 cases of intraventricular hemorrhage. The AI model identified acute subdural/epidural hematoma with area under the curve of 0.973 (95% CI, 0.958–0.984) on thin series and 0.942 (95% CI, 0.921–0.959) on thick series; acute subarachnoid hemorrhage with area under the curve 0.993 (95% CI, 0.984–0.998) on thin series and 0.966 (95% CI, 0.945–0.983) on thick series; intraaxial hemorrhage with area under the curve of 0.969 (95% CI, 0.956–0.980) on thin series and 0.966 (95% CI, 0.953–0.976) on thick series; and intraventricular hemorrhage with area under the curve of 0.987 (95% CI, 0.969–0.997) on thin series and 0.983 (95% CI, 0.968–0.994) on thick series. Each finding had at least 1 operating point with sensitivity and specificity >80%. CONCLUSION The assessed AI model accurately identified intracranial hemorrhage subtypes in this CT data set. Its use could assist the clinical workflow, especially through enabling triage of abnormal CTs.