Inter-modality assessment of medial temporal lobe atrophy in a non-demented population: application of a visual rating scale template across radiologists with varying clinical experience

Abstract

Objectives To assess inter-modality agreement and accuracy for medial temporal lobe atrophy (MTA) ratings across radiologists with varying clinical experience in a non-demented population. Methods Four raters (two junior radiologists and two senior neuroradiologists) rated MTA on CT and MRI scans using Scheltens’ MTA scale. Ratings were compared to a consensus rating by two experienced neuroradiologists for estimation of true positive and negative rates (TPR and TNR) and over- and underestimation of MTA. Inter-modality agreement expressed as Cohen’s κ (dichotomized data), Cohen’s κw, and two-way mixed, single measures, consistency ICC (ordinal data) were determined. Adequate agreement was defined as κ/κw ≥ 0.80 and ICC ≥ 0.80 (significance level at 95% CI ≥ 0.65). Results Forty-nine subjects (median age 72 years, 27% abnormal MTA) with cognitive impairment were included. Only junior radiologists achieved adequate agreement expressed as Cohen’s κ. All raters achieved adequate agreement expressed as Cohen’s κw and ICC. True positive rates varied from 69 to 100% and TNR varied from 85 to 100%. No under- or overestimation of MTA was observed. Ratings did not differ between radiologists. Conclusion We conclude that radiologists with varying experience achieve adequate inter-modality agreement and similar accuracy when Scheltens’ MTA scale is used to rate MTA on a non-demented population. However, TPR varied between radiologists which could be attributed to rating style differences. Key Points • Radiologists with varying experience achieve adequate inter-modality agreement with similar accuracy when Scheltens’ MTA scale is used to rate MTA on a non-demented population. • Differences in rating styles might affect accuracy, this was most evident for senior neuroradiologists, and only junior radiologists achieved adequate agreement on dichotomized (abnormal/normal) ratings. • The use of an MTA scale template might compensate for varying clinical experience which could make it applicable for clinical use.

The role of visual rating and automated brain volumetry in early detection and differential diagnosis of Alzheimer's disease

“…The MTA scale is an important tool for early AD screening and diagnosis, 34 but as a qualitative measure based on 2D MR or CT images, 17 it cannot sensitively identify early or subtle structural changes in AD or MCI 10,35 . Meanwhile, MTA visual scoring relies entirely on professional raters, which lacks objectivity and efficiency 36 . Automatic measurement tools based on artificial intelligence have provided standardized and highly reproducible results 31,37 .…”

“… 10 , 35 Meanwhile, MTA visual scoring relies entirely on professional raters, which lacks objectivity and efficiency. 36 Automatic measurement tools based on artificial intelligence have provided standardized and highly reproducible results. 31 , 37 QMTA, as a quantitative index based on the measurement of 3D images, can observe subtle and systematic changes in brain structure.…”