Evaluation of an artificial intelligence model for identification of intracranial hemorrhage subtypes on computed tomography of the head

Abstract

Importance: Intracranial hemorrhage is a critical finding on computed tomography (CT) of the head. Objective: This study compared the accuracy of an AI model (Annalise Enterprise CTB) to consensus neuroradiologist interpretations in detecting four hemorrhage subtypes: acute subdural/epidural hematoma, acute subarachnoid hemorrhage, intra-axial hemorrhage and intraventricular hemorrhage. Design: A retrospective standalone performance assessment was conducted on datasets of non-contrast CT head cases acquired between 2016 and 2022 for each hemorrhage subtype. Setting: The cases were obtained from five hospitals in the United States. Participants: The cases were obtained from patients aged 18 years or older. The positive cases were selected based on the original clinical reports using natural language processing and manual confirmation. The negative cases were selected by taking the next negative case acquired from the same CT scanner after positive cases. Interventions: Each case was interpreted independently by up to three neuroradiologists to establish consensus interpretations. Each case was then interpreted by the AI model for the presence of the relevant hemorrhage subtype. The neuroradiologists were provided with the entire CT study. The AI model separately received thin (≤1.5mm) and/or thick (>1.5 and ≤5mm) axial series. Results: The four cohorts included 571 cases for acute subdural/epidural hematoma, 310 cases for acute subarachnoid hemorrhage, 926 cases for intra-axial hemorrhage and 199 cases for intraventricular hemorrhage. The AI model identified acute subdural/epidural hematoma with area under the curve (AUC) 0.973 (95% confidence interval (CI), 0.958-0.984) on thin series and 0.942 (95% CI, 0.921-0.959) on thick series; acute subarachnoid hemorrhage with AUC 0.993 (95% CI, 0.984-0.998) on thin series and 0.966 (95% CI, 0.945-0.983) on thick series; intra-axial hemorrhage with AUC 0.969 (95% CI, 0.956-0.980) on thin series and 0.966 (95% CI, 0.953-0.976) on thick series; and intraventricular hemorrhage with AUC 0.987 (95% CI, 0.969-0.997) on thin series and 0.983 (95% CI, 0.968-0.994) on thick series. Each finding had at least one operating point with sensitivity and specificity greater than 80%. Conclusions and Relevance: The assessed AI model accurately identified intracranial hemorrhage subtypes in this CT dataset. Its use could assist the clinical workflow especially through enabling triage of abnormal CTs.