Effect of training and different measurement strategies on the reproducibility of brain MRI lesion load measurements in multiple sclerosis

Abstract

In this study, we evaluated the intra- and interobserver variabilities in measuring lesion load of brain MRI abnormalities present on proton-density scans from patients with MS, using using both manual outlining or a semiautomated local thresholding technique (LTT). We also evaluated how these variabilities were affected by the use of standard rules for lesion load measurements, training, and different measurement strategies. The intraobserver variabilities obtained after establishing rules for lesion load measurements and training were not significantly different from those obtained before any consensus among the observers, both for manual outlining and for the LTT. On the contrary, the interobserver variabilities obtained with manual outlining or the LTT were significantly reduced when rules for lesion load measurements were used. For manual outlining, the intraobserver variability did not significantly change when the measurements were performed after an experienced radiologist identified lesions or when using adjacent slices and the corresponding T2-weighted images as reference for lesion identification. On the contrary, for the LTT, the intraobserver variability was significantly reduced by the use of the radiologic marking. The interobserver variabilities for both manual outlining and the LTT were reduced compared with the free condition when these measurement strategies were used. Our findings demonstrate that both lesion identification and outlining are important sources of variation for MRI lesion load measurements in MS and that there are simple strategies to reduce such variation that might be useful when planning clinical trials.

Multicentre imaging measurements for oncology and in the brain

“…For example, in reporting multiple sclerosis (MS) hypointense lesions from T 2 weighted MRI, differences in qualitative image appearance could be reduced by standardising the acquisition parameters thought to be relevant (repetition time (TR), echo time (TE), slice thickness, type of radiofrequency (RF) coil etc.) and the analysis procedure (by defining rules) [6][7][8][9]. The MAGNIMS (magnetic resonance in multiple sclerosis) study group defined many guidelines for multicentre MR studies in MS [10,11].…”

“…Much work has been published on measuring the total lesion volume in MS [6,7,9]. Standardising the imaging parameters within a range of values and agreeing rules on the lesion segmentation process reduced withincentre variation to a level that is insignificant compared with the natural variation in the disease process.…”

Incorporating Domain Knowledge Into the Fuzzy Connectedness Framework: Application to Brain Lesion Volume Estimation in Multiple Sclerosis

“…Each "observer" actually consisted of two people: an expert neurologist who identified the MS lesions, and an experienced operator (either a neurologist or a technician) who outlined the lesions. The observers followed previously-published guidelines aimed at reducing the interobserver variability as much as possible [28]. However, it should be emphasized that Observer 1 and Observer 2 were operating completely independently on two different continents, and at no time did they confer.…”

“…Using all results from Observer 1, x was fixed by ensuring that in a plot of the lesion volume found using FC versus that found using the contouring method, a regression line had unit slope (i.e., the FC algorithm gave, on average, the same lesion volumes as Observer 1). Observer 1 was chosen because, as will be seen below, they found lower lesion volumes than Observer 2, and the guidelines for lesion volume assessment call for a conservative approach to defining the extent of the lesions [28].…”

Ground Truth in MS Lesion Volumetry – A Phantom Study

“…A phantom with a single lesion type and a fixed number of MS lesions is available from the BrainWeb project. A physical phantom of cylindrical shape with various known dimensions that was placed in an MR scanner is proposed in [10]. Both approaches are important steps towards accurate lesion volumetry but cover only a very limited range of MS lesions.…”