USE-Net: Incorporating Squeeze-and-Excitation blocks into U-Net for prostate zonal segmentation of multi-institutional MRI datasets

Abstract

Prostate cancer is the most common malignant tumors in men but prostate Magnetic Resonance Imaging (MRI) analysis remains challenging. Besides whole prostate gland segmentation, the capability to differentiate between the blurry boundary of the Central Gland (CG) and Peripheral Zone (PZ) can lead to differential diagnosis, since the frequency and severity of tumors differ in these regions. To tackle the prostate zonal segmentation task, we propose a novel Convolutional Neural Network (CNN), called USE-Net, which incorporates Squeeze-and-Excitation (SE) blocks into U-Net, i.e., one of the most effective CNNs in biomedical image segmentation. Especially, the SE blocks are added after every Encoder (Enc USE-Net) or Encoder-Decoder block (Enc-Dec USE-Net). This study evaluates the generalization ability of CNN-based architectures on three T2-weighted MRI datasets, each one consisting of a different number of patients and heterogeneous image characteristics, collected by different institutions. The following mixed scheme is used for training/testing: (i ) training on either each individual dataset or multiple prostate MRI datasets and (ii ) testing on all three datasets with all possible training/testing combinations. USE-Net is compared against three stateof-the-art CNN-based architectures (i.e., U-Net, pix2pix, and Mixed-Scale Dense Network), along with a semi-automatic continuous max-flow model. The results show that training on the union of the datasets generally outperforms training on each dataset separately, allowing for both intra-/crossdataset generalization. Enc USE-Net shows good overall generalization under any training condition, while Enc-Dec USE-Net remarkably outperforms the other methods when trained on all datasets. These findings reveal that the SE blocks' adaptive feature recalibration provides excellent cross-dataset generalization when testing is performed on samples of the datasets used during training. Therefore, we should consider multi-dataset training and SE

MS-Net: Multi-Site Network for Improving Prostate Segmentation With Heterogeneous MRI Data

“…In this section, we take the Joint approach as a reference to compare different multi-site learning methods. In Table IV, USE-Net [19] achieves higher overall performance than the Joint approach. However, the improvement is limited and it still under-performs the Separate approach in site B, indicating that only increasing the network complexity and capacity is insufficient to tackle the inter-site heterogeneity and adequately learn the shared information from multi-site data.…”

“…These works, however, relied heavily on the quality of hand-crafted features. Recently, some works utilized deep learning techniques to brige the inter-site variability [19]- [22]. Rundo et al [19] conducted channel-wise feature calibration to improve the prostate segmentation across heterogeneous datasests.…”

“…Recently, some works utilized deep learning techniques to brige the inter-site variability [19]- [22]. Rundo et al [19] conducted channel-wise feature calibration to improve the prostate segmentation across heterogeneous datasests. To adapt a brain segmentation network to a new modality, Karani et al [20] observed that fine-tuning the BN layers on the new modality while fixing other CNN weights was helpful to preserve the cross-site shared information.…”

“…Meanwhile, images from different sites present heterogeneous in-plane and through-plane resolution, see the fourth column in Table II. In addition, different from the other two datasets, most images (17 out of 19) in Site C are acquired from patients with prostate cancer, leading to a semantic difference in the prostate area. These differences above lead to a visible appearance difference among multi-site data, see Fig.…”

Learning More with Less

“…Also in medical imaging, where the primary problem lies in small and fragmented imaging datasets from various scanners [30], GAN-based DA performs effectively: researchers improved classification by augmentation with noise-to-image GANs (e.g., random noise samples to diverse pathological images) [6] and segmentation with image-to-image GANs (e.g., a benign image with a pathologyconditioning image to a malignant one) [15,32]. Such applications include 256 × 256 brain Magnetic Resonance (MR) image generation for tumor/non-tumor classification [11].…”