Inter-site Variability in Prostate Segmentation Accuracy Using Deep Learning

Abstract

Deep-learning-based segmentation tools are yielding higher reported segmentation accuracies for many medical image segmentation problems. However, inter-site variability in medical image acquisition protocols and quality can challenge the translation of these tools to data from unseen sites. This study quantifies the impact of inter-site variability on the accuracy of deep-learning-based segmentation of the prostate from magnetic resonance (MR) images, and evaluates two strategies for mitigating the performance discrepancies for data from unseen sites: training on multi-site datasets and retraining with data from the unseen site. Using 424 T2-weighted prostate MR images from six sites, we compare the segmentation accuracy of three deep-learning-based networks trained on data from a single site and on various configurations of data from multiple sites. We found that the segmentation accuracy of a single-site network was substantially worse on data from unseen sites than on data from the training site, and that training on multi-site data gives only marginal improvement. However, including as few as 8 subjects from the unseen site, e.g. during commissioning each new clinical system, yields substantial improvement (regaining 75% of the difference).

Designing image segmentation studies: Statistical power, sample size and reference standard quality

“… ), Eq. (7) simplifies to the formula found in our preliminary analysis ( Gibson et al., 2015 ). The term ψ / δ 2 is the squared coefficient of variation of D under the idealized assumption of completely independent voxels (i.e.…”

“…In preliminary work ( Gibson et al., 2015 ), we derived a relationship between statistical power and the quality of a reference standard for a simplified model that cannot account for correlation between voxels, and made a strong assumption that the reference and algorithm segmentation labels are conditionally independent given the high-quality reference standard. In the present paper, we build on our initial work to develop a generalized model that takes into account the correlation between voxels and the statistical dependence between algorithms and reference standards observed in segmentation studies.…”

Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation

“…The large performance degradation of deep models has been observed between different MRI sequences in [6]. Even when using the same MRI sequence, the data distribution can vary in datasets acquired at different centers [5], [29] or different time [4]. Besides MRI images, domain adaptation studies have been performed on cross-site ultrasound datasets [34], [39], X-ray images [35], [43], histopathology applications [40], and optical fundus imaging [17].…”

PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network at Unpaired Cross-Modality Cardiac Segmentation

“…Existing successful practice using deep networks is to train and test the models with the same data source. However, it has been frequently revealed in very recent works, that the models would perform poorly on unseen datasets [7], [10], [43]. Resolving the domain adaptation issue holds great potentials for, applying trained deep learning models to wider clinical use, building more powerful networks using largescale database combing images from multiple sites, and helping to understand how the networks capture the data distributions to make recognition predictions.…”