The use of e-ASPECTS in acute stroke care: validation of method performance compared to the performance of specialists

Abstract

Background: The rapid evaluation of non-contrast-enhanced computed tomography (NCCT) brain scans in patients with anterior stroke symptoms saves time and favors optimal and prompt treatment. e-ASPECTS is a tool that automatically calculates the Alberta Stroke Program Early CT Score (ASPECTS) values, leading to a more accurate and timely image evaluation. Objective: To determine the ability of e-ASPECTS in differentiating images with and without injury. Methods: One-hundred sixteen patients admitted to a stroke unit in a Brazilian tertiary hospital underwent a CT scan at admission and at least one control brain imaging (NCCT or magnetic resonance imaging - MRI) 24 hours after admission. ASPECTS evaluation was performed by three neuroradiologists, three neurologists, and three neurology residents, all blinded to the symptoms and the injury side. The scores were compared to the ground truth, and an ASPECTS score was provided by two independent non blinded evaluators. Sensitivity and specificity were analyzed, and receiver operating characteristic curves, Bland-Altman plots with mean error score, and Matthews correlation coefficients (MCCs) were obtained for ASPECTS scores, assuming values equal to 10 for images without injury and values other than 10 for images with ischemic injury. Results: e-ASPECTS demonstrated similar performance to that of neuroradiologists and neurologists, with an area under the curve of 0.78 and an MCC value of 0.48 in the dichotomous analysis. The sensitivity and specificity of e-ASPECTS were 75% and 73%, respectively. Conclusion: e-ASPECTS is a validated and reliable tool for determining early signs of ischemia in NCCT.

Automated evaluation of ASPECTS from brain computerized tomography of patients with acute ischemic stroke

“…According to the expectations, neurology residents exhibited a lower performance with manual ASPECTS but improved performance with the use of E-ASPECTS. 15 This finding supports the hypothesis that the less experienced the physician, the more beneficial artificial intelligence and the use of E-ASPECTS can be.…”

“…12,13 While manual ASPECTS scoring has a modest interobserver agreement, the use of E-ASPECTS software has increased interrater agreement and the accuracy of infarct extent evaluation in groups of neuroradiologists and neurologists. 14,15 Neuhaus and colleagues studied 16 physicians, including vascular and general neurologists, neuroradiologists, and junior neuroradiologists. In this sample, artificial intelligence improved the reader's perception and accuracy across all levels of experience.…”

External Validation of e‐ASPECTS Software for Interpreting Brain CT in Stroke

“… 29 , 30 As an alternative to accuracy, 3 of 6 studies assessed MCC citing benefits for testing datasets with true positive/negative imbalance. 21 , 31 , 32 Our MCC results are similar (0.34–0.48). For testing agreement between software and experts, our k‐alpha results are like other validated reader‐reliability scoring methods used in 6 studies (median n = 153): kappa (0.25–0.84) 25 , 33 ; intraclass correlation coefficient (0.47–0.87).…”

“…However, 2 studies using summed scores did not control for interdependency between different ASPECTS regions in the same patient 29,30 . As an alternative to accuracy, 3 of 6 studies assessed MCC citing benefits for testing datasets with true positive/negative imbalance 21,31,32 . Our MCC results are similar (0.34–0.48).…”

“…Six studies (median n = 98) include diagnostic accuracy results for e‐ASPECTS with expert reference standards comparable to RITeS: sensitivity 14 to 83%, specificity 57 to 99%, and accuracy 67 to 87%. 21 , 28 , 29 , 30 , 31 , 32 One study used an ASPECTS threshold as we did, 32 the others considered ischemic detection per ASPECTS region for a summed score (ie, 10 × n). However, 2 studies using summed scores did not control for interdependency between different ASPECTS regions in the same patient.…”