CNN-based hierarchical coarse-to-fine segmentation of pelvic CT images for prostate cancer radiotherapy

Abstract

Accurate segmentation of organs-at-risk is important inprostate cancer radiation therapy planning. However, poor soft tissue contrast in CT makes the segmentation task very challenging. We propose a deep convolutional neural network approach to automatically segment the prostate, bladder, and rectum from pelvic CT. A hierarchical coarse-to-fine segmentation strategy is used where the first step generates a coarse segmentation from which an organ-specific region of interest (ROI) localization map is produced. The second step produces detailed and accurate segmentation of the organs. The ROI localization map is generated using a 3D U-net. The localization map helps adjusting the ROI of each organ that needs to be segmented and hence improves computational efficiency by eliminating irrelevant background information. For the fine segmentation step, we designed a fully convolutional network (FCN) by combining a generative adversarial network (GAN) with a U-net. Specifically, the generator is a 3D U-net that is trained to predict individual pelvic structures, and the discriminator is an FCN which fine-tunes the generator predicted segmentation map by comparing it with the ground truth. The network was trained using 100 CT datasets and tested on 15 datasets to segment the prostate, bladder and rectum. The average Dice similarity (mean±SD) of the prostate, bladder and rectum are 0.90±0.05, 0.96±0.06 and 0.91±0.09, respectively, and Hausdorff distances of these three structures are 5.21±1.17, 4.37±0.56 and 6.11±1.47(mm), respectively. The proposed method produces accurate and reproducible segmentation of pelvic structures, which can be potentially valuable for prostate cancer radiotherapy treatment planning.

Deep learning‐based auto segmentation using generative adversarial network on magnetic resonance images obtained for head and neck cancer patients

“…Furthermore, Sultana et al reported that a GAN with a 3D U-net successfully segmented the pelvic region using CT images. 23 The 3D network had the advantage that it obtains more spatial information to use entire image volumes. On the other hand, it has the disadvantage that it requires more training patient data to achieve robust performance.…”

“…Furthermore, Sultana et al. reported that a GAN with a 3D U‐net successfully segmented the pelvic region using CT images 23 . The 3D network had the advantage that it obtains more spatial information to use entire image volumes.…”

U-Net Architectures for Prostate Cancer Radiation Therapy

“…With 3D volumes, voxel size is rarely isotropic. Liu et al (2019), Wang et al (2019), Sultana et al (2020), andMeyer et al (2021) addressed this issue by resampling to isotropic spacing. Le Cun, Kanter, and Solla (1991) showed that standardisation improves convergence properties by forcing the neurons output to the linear region of the activation functions.…”

“…Kingma and Ba (2014) mention that Adam is computationally efficient and with low memory requirements, invariant to diagonal rescale of the gradients, highly appropriate for problems with very noisy/or sparse gradients and requires little tuning. Liu et al (2019), Wang et al (2019), Dai et al (2020), Sultana et al (2020), do not mention the optimizer they used, and A. presented a custom one. The remaining articles used Adam with initial learning rates ranging from 1 × 10 −9 to 1 × 10 −2 .…”

Magnetic Resonance Imaging Based Radiomic Models of Prostate Cancer: A Narrative Review

“…A pragmatic next step would be refining the work to label the ROI through a deep learning approach (e.g., UNet, etc.) [ 127 , 189 , 190 , 191 ] and potentially implementing domain adaptation [ 192 ]. Moreover, there would be a greater amount of data available, as not every imaging data set would require biopsy data as well.…”