MRI FLAIR lesion segmentation in multiple sclerosis: Does automated segmentation hold up with manual annotation?

Abstract

IntroductionMagnetic resonance imaging (MRI) has become key in the diagnosis and disease monitoring of patients with multiple sclerosis (MS). Both, T2 lesion load and Gadolinium (Gd) enhancing T1 lesions represent important endpoints in MS clinical trials by serving as a surrogate of clinical disease activity. T2- and fluid-attenuated inversion recovery (FLAIR) lesion quantification - largely due to methodological constraints – is still being performed manually or in a semi-automated fashion, although strong efforts have been made to allow automated quantitative lesion segmentation. In 2012, Schmidt and co-workers published an algorithm to be applied on FLAIR sequences. The aim of this study was to apply the Schmidt algorithm on an independent data set and compare automated segmentation to inter-rater variability of three independent, experienced raters.MethodsMRI data of 50 patients with RRMS were randomly selected from a larger pool of MS patients attending the MS Clinic at the Brain and Mind Centre, University of Sydney, Australia. MRIs were acquired on a 3.0T GE scanner (Discovery MR750, GE Medical Systems, Milwaukee, WI) using an 8 channel head coil. We determined T2-lesion load (total lesion volume and total lesion number) using three versions of an automated segmentation algorithm (Lesion growth algorithm (LGA) based on SPM8 or SPM12 and lesion prediction algorithm (LPA) based on SPM12) as first described by Schmidt et al. (2012). Additionally, manual segmentation was performed by three independent raters. We calculated inter-rater correlation coefficients (ICC) and dice coefficients (DC) for all possible pairwise comparisons.ResultsWe found a strong correlation between manual and automated lesion segmentation based on LGA SPM8, regarding lesion volume (ICC = 0.958 and DC = 0.60) that was not statistically different from the inter-rater correlation (ICC = 0.97 and DC = 0.66). Correlation between the two other algorithms (LGA SPM12 and LPA SPM12) and manual raters was weaker but still adequate (ICC = 0.927 and DC = 0.53 for LGA SPM12 and ICC = 0.949 and DC = 0.57 for LPA SPM12). Variability of both manual and automated segmentation was significantly higher regarding lesion numbers.ConclusionAutomated lesion volume quantification can be applied reliably on FLAIR data sets using the SPM based algorithm of Schmidt et al. and shows good agreement with manual segmentation.

Value of fluid‐attenuated inversion recovery MRI data analyzed by the lesion segmentation toolbox in amyotrophic lateral sclerosis

“…This finding is consistent with Vazques-Costa et al 23 reporting more pronounced FLAIR signal change of the CST in bulbar-compared with limb-onset patients. Total lesion volumes in ALS patients were comparable to that of healthy controls and much lower than in other neurological diseases such as MS. 21 These findings are consistent with previous studies reporting the dominant role of frequency rather than volumes of FLAIR alterations in ALS. 13,14 FLAIR lesions in ALS patients were most prominent in the corona radiata and body and genu of the CC.…”

“…Values of interest like TLV (total lesion volume in ml) and TLN (number of lesions) were extracted by thresholding individual lesion probability maps. Threshold kappa was set to κ = 0.5, as this threshold was recommended for LPA SPM12 use . Binary lesion maps thresholded by κ enabled the visualization of individual patients' FLAIR lesions.…”

“…Threshold kappa was set to κ = 0.5, as this threshold was recommended for LPA SPM12 use. 21 Binary lesion maps thresholded by κ enabled the visualization of individual patients' FLAIR lesions. T 1 data are not required for LPA use.…”

Blood–brain barrier permeability measured using dynamic contrast‐enhanced magnetic resonance imaging: a validation study

“…The automated lesion detection algorithm is validated for 3D FLAIR (Egger et al . ). Thus we could have inadvertently included lesional tissue in NAWM during segmentation.…”

“…Fourthly, the FLAIR sequences used were 2D, which is inferior to 3D in the detection of MS lesions (Bink et al 2006). The automated lesion detection algorithm is validated for 3D FLAIR (Egger et al 2017). Thus we could have inadvertently included lesional tissue in NAWM during segmentation.…”