Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach

Abstract

In this paper, we present a novel automated method for White Matter (WM) lesion segmentation of Multiple Sclerosis (MS) patient images. Our approach is based on a cascade of two 3D patch-wise convolutional neural networks (CNN). The first network is trained to be more sensitive revealing possible candidate lesion voxels while the second network is trained to reduce the number of misclassified voxels coming from the first network. This cascaded CNN architecture tends to learn well from small sets of training data, which can be very interesting in practice, given the difficulty to obtain manual label annotations and the large amount of available unlabeled Magnetic Resonance Imaging (MRI) data. We evaluate the accuracy of the proposed method on the public MS lesion segmentation challenge MICCAI2008 dataset, comparing it with respect to other state-of-the-art MS lesion segmentation tools. Furthermore, the proposed method is also evaluated on two private MS clinical datasets, where the performance of our method is also compared with different recent public available state-of-the-art MS lesion segmentation methods. At the time of writing this paper, our method is the best ranked approach on the MICCAI2008 challenge, outperforming the rest of 60 participant methods when using all the available input modalities (T1-w, T2-w and FLAIR), while still in the top-rank (3rd position) when using only T1-w and FLAIR modalities. On clinical MS data, our approach exhibits a significant increase in the accuracy segmenting of WM lesions when compared with the rest of evaluated methods, highly correlating (r ≥ 0.97) also with the expected lesion volume.

Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks

“…Compared to DSC which weighs recall and precision equally, and the ROC analysis, we consider the area under the PR curves (APR, shown in Figure 2) the most reliable performance metric for such highly skewed data [8,1]. To put the work in context, we reported average DSC, F 2 , and APR scores (equal to 56.4, 57.3, and 56.0, respectively), which indicate that our approach performed very well compared to the latest results in MS lesion segmentation [6,20]. We did not conduct a direct comparison in the application domain, however, as this paper intended to provide proof-of-concept on the effect and usefulness of the Tverky loss layer (and F β scores) in deep learning.…”

“…We tested our FCN with Tversky loss layer to segment multiple sclerosis (MS) lesions [6,20]. T1-weighted, T2-weighted, and FLAIR MRI of 15 subjects were used as input, where we used two-fold cross-validation for training and testing.…”

“…A common approach to account for data imbalance, especially in voxelwise methods, is to extract equal training samples from each class [20]. The downsides of this approach are that it does not use all the information content of the images and may bias towards rare classes.…”

“…The downsides of this approach are that it does not use all the information content of the images and may bias towards rare classes. Hierarchical training [5,21,20] and retraining [9] have been proposed as alternative strategies but they can be prone to overfitting and sensitive to the state of the initial classifiers [10]. Recent training methods for FCNs resorted to loss functions based on sample re-weighting [2,10,12,16,18], where lesion regions, for example, are given more importance than non-lesion regions during training.…”

“…We show how adjusting the hyperparameters of this index allow placing emphasis on false negatives in training a network that generalizes and performs well in highly imbalanced data as it leads to high sensitivity, Dice, F 2 score, and the area under the precision-recall (PR) curve [1] in the test set. To this end, we adopt a 3D FCN, based on the U-net architecture, with a Tversky loss layer, and test it in the challenging multiple sclerosis lesion detection problem on multi-channel MRI [6,20]. The ability to train a network for higher sensitivity (recall) in the expense of acceptable decrease in precision is crucial in many medical image segmentation tasks such as lesion detection.…”