Reproducibility of Brain MRI Lesion Volume Measurements in Multiple Sclerosis Using a Local Thresholding Technique: Effects of Formal Operator Training

Abstract

The assessment of lesion load (LL) on brain magnetic resonance imaging (MRI) scans from patients with multiple sclerosis (MS) is widely used to monitor disease evolution, natural or modified by treatments. In this study, we evaluated the effect of formal operator training on the intra- and inter-observer reproducibility of LL measurements obtained by several operators in a setting similar to that of clinical trials. Proton-density (PD)-weighted, unenhanced and enhanced T1-weighted brain MRI scans were obtained from 10 MS patients. Five naive technicians assessed LL on these images, using a semiautomated local thresholding technique for lesion segmentation and marked hardcopies as a reference. Measurements were performed twice before and twice after a 20-hour operator training. Mean intra-observer measurement coefficient of variations (COV) before and after the training were 3.1 and 1.6% for PD-weighted LL, 4.3 and 1.8% for unenhanced T1-weighted LL (p < 0.001), 4.9 and 2.0% for enhanced T1-weighted LL (p = 0.002). Mean inter-observer COV were significantly reduced after training (from 10.0 to 5.6% for PD-weighted, from 11.0 to 7.3% for unenhanced T1-weighted and from 16.0 to 6.8% for enhanced T1-weighted LL). Our data indicate that LL assessment on serial MRI scans from MS patients performed by technicians, using a local thresholding technique for lesion segmentation, is characterized by low measurement variability which may be significantly improved by a short and cost-effective training.

The Role of Centralized Reading of Endoscopy in a Randomized Controlled Trial of Mesalamine for Ulcerative Colitis

“…Central adjudication of images in other clinical circumstances has become the gold standard for this reason. [29][30][31][32][33][34][35][36][37][38] In summary, we have shown that mesalamine (800 mg) administered twice daily for 10 weeks at a dosage of 4.8 g/day is effective for induction of clinical and endoscopic remission in patients with mildly to moderately active UC. Importantly, we also show that evaluation of endoscopic eligibility criteria by a central reader has the potential to improve the efficiency of randomized controlled trials of induction therapy in UC by minimizing inclusion of patients with low endoscopic disease activity that increases placebo rates.…”

Automated segmentation and measurement of global white matter lesion volume in patients with multiple sclerosis

“…These operator-assisted procedures such as local thresholding have been successfully employed (7); however they require time and are indeed associated with some degree of intra and interoperator variability (8,9). Therefore, it is obvious that the development of a tool that would permit automated measurement of brain lesion load in MS is an actively pursued goal by several research groups.…”

Improved operator agreement and efficiency using the minimum area contour change method for delineation of hyperintense multiple sclerosis lesions on FLAIR MRI

“…Large agreement differences often exist between operators drawing regions of interest (ROIs) of hyperintense lesions on Fluid Attenuated Inversion Recovery (FLAIR) MRI brain scans of patients with multiple sclerosis (MS) despite training and standards designed to minimize inter-operator variability [1-4]. These differences can be broken down into two categories: a) lesion outline disagreement and b) lesion detection disagreement.…”