Multi-branch convolutional neural network for multiple sclerosis lesion segmentation

Abstract

In this paper, we present an automated approach for segmenting multiple sclerosis (MS) lesions from multi-modal brain magnetic resonance images. Our method is based on a deep end-to-end 2D convolutional neural network (CNN) for slice-based segmentation of 3D volumetric data. The proposed CNN includes a multi-branch downsampling path, which enables the network to encode information from multiple modalities separately. Multi-scale feature fusion blocks are proposed to combine feature maps from different modalities at different stages of the network. Then, multi-scale feature upsampling blocks are introduced to upsize combined feature maps to leverage information from lesion shape and location. We trained and tested the proposed model using orthogonal plane orientations of each 3D modality to exploit the contextual information in all directions. The proposed pipeline is evaluated on two different datasets: a private dataset including 37 MS patients and a publicly available dataset known as the ISBI 2015 longitudinal MS lesion segmentation challenge dataset, consisting of 14 MS patients. Considering the ISBI challenge, at the time of submission, our method was amongst the top performing solutions. On the private dataset, using the same array of performance metrics as in the ISBI challenge, the proposed approach shows high improvements in MS lesion segmentation compared with other publicly available tools.

Multiple sclerosis cortical and WM lesion segmentation at 3T MRI: a deep learning method based on FLAIR and MP2RAGE

“…For comparison, we evaluated two state-of-the-art MS WML segmentation methods publicly available: LST-LGA is an unsupervised lesion growth algorithm ( Schmidt et al, 2012 ) implemented in the LST toolbox version 3.0.0 ( LST, 2020 ) for Statistical Parametric Mapping (SPM). LST-LGA has been widely evaluated in the context of MS WML segmentation and used as comparison with more recent approaches ( Aslani et al, 2019 , Valverde et al, 2017 , Roy et al, 2018 ). In a nutshell, the algorithm performs an initial bias field correction and affine registration of the T1 image (in our case the MP2RAGE) to the FLAIR, and then proceed with the lesion segmentation.…”

“… nicMSlesions is a state-of-the-art deep learning WML segmentation method ( Valverde et al, 2017 , Valverde et al, 2019 ). Having reached an excellent performance in a MS lesion segmentation challenge ( Carass et al, 2017 ), it is now a common method to compare with ( Aslani et al, 2019 , Weeda et al, 2019 , Roy et al, 2018 ). This method selects lesion candidates’ voxels based on the FLAIR contrast and extracts 11x11x11 patches around them.…”

“…Specifically, several convolutional neural network (CNN) architectures have been tailored for the segmentation of MS WMLs ( Kaur et al, 2020 ). Some of them employ 2D convolutional layers ( Aslani et al, 2019 , Roy et al, 2018 ), whereas others employ 3D convolutional layers to incorporate information from all three spatial directions simultaneously ( Hashemi et al, 2019 , La Rosa et al, 2019 , Valverde et al, 2017 , Valverde et al, 2019 ). The clear edge these methods have over classical approaches is the capability of automatically extracting the relevant features for the task.…”

Dynamic Changes of Multiple Sclerosis Lesions on T2-FLAIR MRI using Digital Image Processing

“…Aslani et al [27] introduced an automated method for segmenting MS lesions from multi-modal brain magnetic resonance images. Their approach based on a deep-end to end 2D CNN.…”

An iterative multi‐path fully convolutional neural network for automatic cardiac segmentation in cine MR images

“…[15][16][17] But as taking 3D context information into consideration, 3D methods can maintain the 3D consistency between the segmentation of different slices and should be theoretically more consistent and accurate than 2D methods. However, in the context of cardiac image segmentation, due to the low through-plane resolution characteristic of cardiac MRI and the shortcomings of 3D methods such as reduction of training images and high risk of overfitting, 15,18 the segmentation performance of 3D methods may be limited to some extent. For instance, the previous work [19][20][21] evaluated on the automatic cardiac diagnosis challenge (ACDC) dataset indicated that the proposed 3D models did not meet the expectations in performance improvement over the corresponding 2D models.…”