PSACNN: Pulse sequence adaptive fast whole brain segmentation

Abstract

With the advent of convolutional neural networks (CNN), supervised learning methods are increasingly being used for whole brain segmentation. However, a large, manually annotated training dataset of labeled brain images required to train such supervised methods is frequently difficult to obtain or create. In addition, existing training datasets are generally acquired with a homogeneous magnetic resonance imaging (MRI) acquisition protocol. CNNs trained on such datasets are unable to generalize on test data with different acquisition protocols. Modern neuroimaging studies and clinical trials are necessarily multi-center initiatives with a wide variety of acquisition protocols. Despite stringent protocol harmonization practices, it is very difficult to standardize the gamut of MRI imaging parameters across scanners, field strengths, receive coils etc., that affect image contrast. In this paper we propose a CNN-based segmentation algorithm that, in addition to being highly accurate and fast, is also resilient to variation in the input acquisition. Our approach relies on building approximate forward models of pulse sequences that produce a typical test image. For a given pulse sequence, we use its forward model to generate plausible, synthetic training examples that appear as if they were acquired in a scanner with that pulse sequence. Sampling over a wide variety of pulse sequences results in a wide variety of augmented training examples that help build an image contrast invariant model. Our method trains a single CNN that can segment input MRI images with acquisition parameters as disparate as T 1 -weighted and T 2 -weighted contrasts with only T 1 -weighted training data. The segmentations generated are highly accurate with state-of-the-art results (overall Dice overlap= 0.94), with a fast run time (≈ 45 seconds), and consistent across a wide range of acquisition protocols.

FastSurfer - A fast and accurate deep learning based neuroimaging pipeline

“…The task of whole-brain segmentation in particular is challenging due to the complex 3D architecture and spatial dependency between slices, the large number of labels, the size of the scanning volumes (memory requirements), and variability across scanners and subjects. While several deep learning based approaches have been proposed for specific tasks, such as tumor segmentation ( Rani and Vashisth ; Dong et al, 2017 ; Arunachalam and Savarimuthu, 2017 ; Havaei et al, 2017 ; Amin et al, 2018 ; Brainless Glioma , 2018 ), brain lesion segmentation ( Kamnitsas et al, 2017 ; Varghese et al, 2017 ; Rezaei et al, 2017 ; Roa-Barco et al, 2017 ; Chen and Konukoglu, 2018 ), MR image reconstruction ( Jin et al, 2017 ; Mardani et al, 2019 ; Schlemper et al, 2018 ; Yang et al, 2018 ; Dedmari et al, 2018 ), prediction of brain related diseases and their progression ( Payan and Montana, 2015 ; Qi and Tejedor, 2016 ; Hosseini-Asl et al, 2016 ; Lee et al Kim ) or segmentation of a smaller number of brain (sub-)structures ( Zhang et al, 2015 ; Akkus et al, 2017 ; Milletari et al, 2017 ; Fedorov et al, 2017 ; Dolz et al, 2018 ; Thyreau et al, 2018 ; Chen et al, 2018 ; Nogovitsyn et al, 2019 ; Li et al, 2019 ; Sun et al, 2019 ; Ito et al, 2019 ) full brain segmentation into more than 25 classes has - so far - only been achieved by a few groups ( de Brêbisson and Montana, 2015 ; Moeskops et al, 2016 ; Mehta et al, 2017 ; Wachinger et al, 2018 ; Roy et al, 2017 , 2019 ; Jog et al, 2019 ; Huo et al, 2019 ; Coupé et al, 2019 ) - yet with the exception of ( Roy et al, 2019 ) only with direct comparison of segmentation accuracy on a test-set, lacking extensive validations, e.g., of reliability and sensitivity to real neuroanatomical effects.…”

“…Most of these brain segmentation networks were trained on extracted 3D patches ( de Brêbisson and Montana, 2015 ; Wachinger et al, 2018 ; Mehta et al, 2017 ) or 2D slices ( Moeskops et al, 2016 ; Roy et al, 2017 , 2019 ; Jog et al, 2019 ). Both approaches loose spatial information critical for correct classification of a given structure.…”

End-to-end semantic segmentation of personalized deep brain structures for non-invasive brain stimulation

“…With deep learning network architectures, several automatic features can be observed by the network with no need of prior hand-crafted design . Recently, several network designs are presented for brain segmentation (Chen et al, 2018;Dolz et al, 2019;Jog et al, 2019;Khalili et al, 2019;Wachinger et al, 2018) and deep brain regions (Dolz et al, 2018;Kushibar et al, 2018;Roy et al, 2019;Ryu et al, 2019). Reviews on brain structure segmentation in MRI can be found in González-Villá et al (2016) with an emphasis on deep learning approaches in Akkus et al (2017).…”

Brain Morphometry Estimation: From Hours to Seconds Using Deep Learning

“…Regarding brain anatomy, promising results in the application of deep learning-based models were observed for the segmentation of tissue classes and subcortical structures (33)(34)(35)(36)(37)(38). The challenge of having access to enough labeled data for training is addressed by semi-supervised (39) and unsupervised (40) approaches or data augmentation strategies simulating diverse pulse sequences (41). While these segmentation-based methods enable calculation of volumes in a timely fashion, none of them provide thickness or curvature measures of the cortex.…”