MRI pulse sequence integration for deep‐learning‐based brain metastases segmentation

Abstract

Purpose Magnetic resonance (MR) imaging is an essential diagnostic tool in clinical medicine. Recently, a variety of deep‐learning methods have been applied to segmentation tasks in medical images, with promising results for computer‐aided diagnosis. For MR images, effectively integrating different pulse sequences is important to optimize performance. However, the best way to integrate different pulse sequences remains unclear. In addition, networks trained with a certain subset of pulse sequences as input are unable to perform when given a subset of those pulse sequences. In this study, we evaluate multiple architectural features and characterize their effects in the task of metastasis segmentation while creating a method to robustly train a network to be able to work given any strict subset of the pulse sequences available during training. Methods We use a 2.5D DeepLabv3 segmentation network to segment metastases lesions on brain MR's with four pulse sequence inputs. To study how we can best integrate MR pulse sequences for this task, we consider (1) different pulse sequence integration schemas, combining our features at early, middle, and late points within a deep network, (2) different modes of weight sharing for parallel network branches, and (3) a novel integration level dropout layer, which will allow the networks to be robust to performing inference on input with only a subset of pulse sequences available at the training. Results We find that levels of integration and modes of weight sharing that favor low variance work best in our regime of small amounts of training data (n = 100). By adding an input‐level dropout layer, we could preserve the overall performance of these networks while allowing for inference on inputs with missing pulse sequences. We illustrate not only the generalizability of the network but also the utility of this robustness when applying the trained model to data from a different center, which does not use the same pulse sequences. Finally, we apply network visualization methods to better understand which input features are most important for network performance. Conclusions Together, these results provide a framework for building networks with enhanced robustness to missing data while maintaining comparable performance in medical imaging applications.

2.5D and 3D segmentation of brain metastases with deep learning on multinational MRI data

“…Compared to previous works on the same Stanford cohort (Grøvik et al, 2020 ) and OUH cohort (Grøvik et al, 2021 ; Yi et al, 2021 ), our model archives a similar or improved per patient detection sensitivity with 0.88/0.83 compared to their previously published average sensitivity of 0.83, whilst reducing the total number of false positives from an average of 8.3 to 6.2/3.2 for the 2.5D and 3D networks, respectively. A similar, but more pronounced effect can be seen for the OUH cohort where the number of false positives was reduced from an average of 12.3 (Grøvik et al, 2021 ) to 1.0 or 0.4 per patient for the 2.5D and 3D networks, respectively.…”

“…The high-resolution network for 2.5D and 3D segmentation (Wang et al, 2019) was adopted in combination with mixup augmentation (Zhang et al, 2017), and deep supervision (Wang et al, 2015). We demonstrate that the proposed 2.5D and 3D deep learning-based segmentation models can successfully be used for segmentation on two separate clinical protocols, whilst reducing the number of false positives previously reported for both cohorts without reducing the number of successfully detected metastases (Grøvik et al, 2020(Grøvik et al, , 2021Yi et al, 2021). Method performance was evaluated by adopting the nnU-Net (Isensee et al, 2020a) framework as a comparative benchmark.…”

Stratified assessment of an FDA-cleared deep learning algorithm for automated detection and contouring of metastatic brain tumors in stereotactic radiosurgery

“…Artificial intelligence (AI) has demonstrated promise in addressing these issues. With the goal of improving efficiency and standardization, machine learning models have recently been developed for automated detection and segmentation of metastatic brain tumors [2,[5][6][7][8][9][10][11][12]. However, the published literature thus far is comprised of technical proof-of-concepts in which the model is tested on small, limited sample sizes, and/or it is not readily deployable to the clinic.…”