Region-specific agreement in ASPECTS estimation between neuroradiologists and e-ASPECTS software

Abstract

Background and purposeThe Alberta Stroke Program Early CT Score (ASPECTS) is a widely used measure of ischemic change on non-contrast CT. Although predictive of long-term outcome, ASPECTS is limited by its modest interobserver agreement. One potential solution to this is the use of machine learning strategies, such as e-ASPECTS, to detect ischemia. Here, we compared e-ASPECTS with manual scoring by experienced neuroradiologists for all 10 individual ASPECTS regions.Materials and methodsWe retrospectively reviewed 178 baseline non-contrast CT scans from patients with acute ischemic stroke undergoing endovascular thrombectomy. All scans were reviewed by two independent neuroradiologists with a third reader arbitrating disagreements for a consensus read. Each ASPECTS region was scored individually. All scans were then evaluated using a machine learning-based software package (e-ASPECTS, Brainomix). Interobserver agreement between readers and the software for each region was calculated with a kappa statistic.ResultsThe median ASPECTS was 9 for manual scoring and 8.5 for e-ASPECTS, with an overall agreement of κ=0.248. Regional agreement varied from κ=0.094 (M1) to κ=0.555 (lentiform), with better performance in subcortical regions. When corrected for the low number of infarcts in any given region, prevalence-adjusted bias-adjusted kappa ranged from 0.483 (insula) to 0.888 (M3), with greater agreement for cortical areas. Intraclass correlation coefficients were between 0.09 (M1) and 0.556 (lentiform).ConclusionManual scoring and e-ASPECTS had fair agreement in our dataset on a per-region basis. This warrants further investigation using follow-up scans or MRI as the gold standard measure of true ASPECTS.

Improved Reliability of Automated ASPECTS Evaluation Using Iterative Model Reconstruction from Head CT Scans

“…This constitutes a considerable improvement in software robustness compared to an earlier study (using software version 4.9), which reported 32 failed cases in 226 cases analyzed (86% success rate) 14 . Other studies showed good agreement of total ASPECTS between automated software and human readers, but found higher variance in region‐specific agreement 22,23 …”

“…14 Other studies showed good agreement of total ASPECTS between automated software and human readers, but found higher variance in regionspecific agreement. 22,23 To eliminate the human factor, we also assessed how IMR impacts objective image quality using automated software as a quantitative benchmark. Interestingly, the agreement of software rating could be improved by using IMR with a slice thickness of 0.9 mm.…”

Total and regional ASPECT score for non-contrast CT, CT angiography, and CT perfusion: inter-rater agreement and its association with the final infarction in acute ischemic stroke patients

“…Recently, automated ASPECTS software based on NCCT has been reported to not be inferior to neuroradiologists in determining early ischemic changes and has been used to assist in decision making (20,21). However, its low sensitivity to detect early ischemic changes in the internal capsule and deep regions has limited its application value (22,23). Based on our results, we think that further attempts to automatically and electronically calculate ASPECTS based on CTP-CBV map may be a potential approach to improve the delineation of early ischemic extent in patients with AIS.…”

Deep learning derived automated ASPECTS on non‐contrast CT scans of acute ischemic stroke patients

“…The four subcortical regions include caudate (C), lentiform (L), internal capsule (IC), and insular (I). Then, if the left and right sides of one region show signs of ischemic stroke, such as hypodensity, gray‐white distinction, and focal swelling, 1 point is deducted from the initial score of 10 points (Neuhaus et al, 2020 ). ASPECTS scores were found correlated with clinical outcomes, where the American Heart Association/American Stroke Association (AHA/ASA) guidelines recommend selecting patients with ASPECTS 6 for intravascular thrombectomy (Powers et al, 2019 ).…”

“…Hoelter et al concluded that among the three most common scoring software on the market (Syngo.via Frontier ASPECT Score Prototype V2, Brainomix e‐ASPECTS and RAPID ASPECTS), the highest correlation ( r = 0.871, p < .001) was found between expert reading and Brainomix (e‐ASPECTS software), followed by Frontier V2, then RAPID (Hoelter et al, 2020 ). Neuhaus et al compared the results of the automated software e‐ASPECTS with ground truth acquired from experts and obtained fair agreement in 178 baseline NCCT scans (Neuhaus et al, 2020 ). Brinjikji et al separately compare the inter‐class concordance of ASPECTS scores given by 16 experts for 60 patients in the condition with and without the e‐ASPECTS aid in NCCT and follow‐up CT, and they found the e‐ASPECTS software could improve the accuracy of ischemic prediction (Brinjikji et al, 2021 ).…”