A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols

Karani, Neerav; Chaitanya, Krishna; Baumgartner, Christian F.; Konukoğlu, Ender

doi:10.1007/978-3-030-00928-1_54

Cited by 108 publications

(109 citation statements)

References 24 publications

(38 reference statements)

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…Existing successful practice using deep networks is to train and test the models with the same data source. However, it has been frequently revealed in very recent works, that the models would perform poorly on unseen datasets [7], [10], [43]. Resolving the domain adaptation issue holds great potentials for, applying trained deep learning models to wider clinical use, building more powerful networks using largescale database combing images from multiple sites, and helping to understand how the networks capture the data distributions to make recognition predictions.…”

Section: Discussionmentioning

confidence: 99%

PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network at Unpaired Cross-Modality Cardiac Segmentation

et al. 2019

View full text Add to dashboard Cite

Deep convolutional networks have demonstrated the state-of-the-art performance on various medical image computing tasks. Leveraging images from different modalities for the same analysis task holds clinical benefits. However, the generalization capability of deep models on test data with different distributions remain as a major challenge. In this paper, we propose the PnP-AdaNet (plug-and-play adversarial domain adaptation network) for adapting segmentation networks between different modalities of medical images, e.g., MRI and CT. We propose to tackle the significant domain shift by aligning the feature spaces of source and target domains in an unsupervised manner. Specifically, a domain adaptation module flexibly replaces the early encoder layers of the source network, and the higher layers are shared between domains. With adversarial learning, we build two discriminators whose inputs are respectively multi-level features and predicted segmentation masks. We have validated our domain adaptation method on cardiac structure segmentation in unpaired MRI and CT. The experimental results with comprehensive ablation studies demonstrate the excellent efficacy of our proposed PnP-AdaNet. Moreover, we introduce a novel benchmark on the cardiac dataset for the task of unsupervised cross-modality domain adaptation. We will make our code and database publicly available, aiming to promote future studies on this challenging yet important research topic in medical imaging.

show abstract

Section: Discussionmentioning

confidence: 99%

PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network at Unpaired Cross-Modality Cardiac Segmentation

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Conversely, it also restricted the adaptation capacity to the new domain. Karani et al suggested that learning batch normalization parameters for each scanner and sharing the convolutional filters between all scanners addressed the distribution shift among scanners [91]. The experiment showed this strategy can be adapted to new scanners or protocols with only a few (≈ 4) labelled images and without degrading performance on the previous scanners.…”

Section: Choosing and Training Modelsmentioning

confidence: 99%

Artificial intelligence in glioma imaging: challenges and advances

et al. 2020

View full text Add to dashboard Cite

Primary brain tumors including gliomas continue to pose significant management challenges to clinicians. While the presentation, the pathology, and the clinical course of these lesions is variable, the initial investigations are usually similar. Patients who are suspected to have a brain tumor will be assessed with computed tomography (CT) and magnetic resonance imaging (MRI). The imaging findings are used by neurosurgeons to determine the feasibility of surgical resection and plan such an undertaking. Imaging studies are also an indispensable tool in tracking tumor progression or its response to treatment. As these imaging studies are non-invasive, relatively cheap and accessible to patients, there have been many efforts over the past two decades to increase the amount of clinically-relevant information that can be extracted from brain imaging. Most recently, artificial intelligence (AI) techniques have been employed to segment and characterize brain tumors, as well as to detect progression or treatment-response. However, the clinical utility of such endeavours remains limited due to challenges in data collection and annotation, model training, and in the reliability of AI-generated information.We provide a review of recent advances in addressing the above challenges. First, to overcome the challenge of data paucity, different image imputation and synthesis techniques along with annotation collection efforts are summarized. Next, various training strategies are presented to meet multiple desiderata, such as model performance, generalization ability, data privacy protection, and learning with sparse annotations. Finally, standardized performance evaluation and model interpretability methods have been reviewed. We believe that these technical approaches will facilitate the development of a fully-functional AI tool in the clinical care of patients with gliomas.

show abstract

“…Despite the powerful local and global context that CNNs generally provide, they are more vulnerable to subtle contrast differences between training and test MRI images than model-based and MALF-based methods. A recent work by Karani et al (2018) describes a brain segmentation CNN trained to consistently segment multi-scanner and multi-protocol data. This framework adds a new set of batch normalization parameters in the network as it encounters training data from a new acquisition protocol.…”

Section: Prior Workmentioning

confidence: 99%

PSACNN: Pulse sequence adaptive fast whole brain segmentation

et al. 2019

View full text Add to dashboard Cite

With the advent of convolutional neural networks (CNN), supervised learning methods are increasingly being used for whole brain segmentation. However, a large, manually annotated training dataset of labeled brain images required to train such supervised methods is frequently difficult to obtain or create. In addition, existing training datasets are generally acquired with a homogeneous magnetic resonance imaging (MRI) acquisition protocol. CNNs trained on such datasets are unable to generalize on test data with different acquisition protocols. Modern neuroimaging studies and clinical trials are necessarily multi-center initiatives with a wide variety of acquisition protocols. Despite stringent protocol harmonization practices, it is very difficult to standardize the gamut of MRI imaging parameters across scanners, field strengths, receive coils etc., that affect image contrast. In this paper we propose a CNN-based segmentation algorithm that, in addition to being highly accurate and fast, is also resilient to variation in the input acquisition. Our approach relies on building approximate forward models of pulse sequences that produce a typical test image. For a given pulse sequence, we use its forward model to generate plausible, synthetic training examples that appear as if they were acquired in a scanner with that pulse sequence. Sampling over a wide variety of pulse sequences results in a wide variety of augmented training examples that help build an image contrast invariant model. Our method trains a single CNN that can segment input MRI images with acquisition parameters as disparate as T 1 -weighted and T 2 -weighted contrasts with only T 1 -weighted training data. The segmentations generated are highly accurate with state-of-the-art results (overall Dice overlap= 0.94), with a fast run time (≈ 45 seconds), and consistent across a wide range of acquisition protocols.

show abstract

A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols

Cited by 108 publications

References 24 publications

PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network at Unpaired Cross-Modality Cardiac Segmentation

PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network at Unpaired Cross-Modality Cardiac Segmentation

Artificial intelligence in glioma imaging: challenges and advances

PSACNN: Pulse sequence adaptive fast whole brain segmentation

Contact Info

Product

Resources

About