Cross-Modality Knowledge Transfer for Prostate Segmentation from CT Scans

Liu, Yucheng; Khosravan, Naji; Stember, Joseph N.; Shoag, Jonathan; Bağcı, Ulaş; Jambawalikar, Sachin

doi:10.1007/978-3-030-33391-1_8

Cited by 15 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These datasets were resampled to the same target spacing (2, 2, 2) and embedded into a 256 × 256 × 256 3D volumetric space [35]. After normalizing and window leveling [−200, 250] [36][37][38][39], to enhance the contrast and texture of soft tissue, the foreground of input voxels was selected from the background by an intersection with mask voxels images using MATLAB R2022a. To increase the amount of data for training the network, we augmented the CT images…”

Section: Patient Cohorts and Data Pre-processingmentioning

confidence: 99%

Empowering Vision Transformer by Network Hyper-Parameter Selection for Whole Pelvis Prostate Planning Target Volume Auto-Segmentation

Cho,

Lee,

Kim

et al. 2023

Cancers

View full text Add to dashboard Cite

U-Net, based on a deep convolutional network (CNN), has been clinically used to auto-segment normal organs, while still being limited to the planning target volume (PTV) segmentation. This work aims to address the problems in two aspects: 1) apply one of the newest network architectures such as vision transformers other than the CNN-based networks, and 2) find an appropriate combination of network hyper-parameters with reference to recently proposed nnU-Net (“no-new-Net”). VT U-Net was adopted for auto-segmenting the whole pelvis prostate PTV as it consisted of fully transformer architecture. The upgraded version (v.2) applied the nnU-Net-like hyper-parameter optimizations, which did not fully cover the transformer-oriented hyper-parameters. Thus, we tried to find a suitable combination of two key hyper-parameters (patch size and embedded dimension) for 140 CT scans throughout 4-fold cross validation. The VT U-Net v.2 with hyper-parameter tuning yielded the highest dice similarity coefficient (DSC) of 82.5 and the lowest 95% Haussdorff distance (HD95) of 3.5 on average among the seven recently proposed deep learning networks. Importantly, the nnU-Net with hyper-parameter optimization achieved competitive performance, although this was based on the convolution layers. The network hyper-parameter tuning was demonstrated to be necessary even for the newly developed architecture of vision transformers.

show abstract

Section: Patient Cohorts and Data Pre-processingmentioning

confidence: 99%

Empowering Vision Transformer by Network Hyper-Parameter Selection for Whole Pelvis Prostate Planning Target Volume Auto-Segmentation

Cho,

Lee,

Kim

et al. 2023

Cancers

View full text Add to dashboard Cite

show abstract

“…Incorporating data augmentation and SSIM as cycle-consistency loss they manage to achieve a difference in Dice coefficient between their synthetic CT images and real CT images of 0.07. 11 Similar works on cross-modality transfer using CycleGAN for segmentation where done in area of lung tumor segmentation 12 and segmentation of the parotid glands. 13 One great difference between the mentioned imagetranslation methods and GIN augmentation is, that GIN augmentation removes the need to train a whole new deep learning model and can be employed directly as a data augmentation method for the segmentation model.…”

Section: Introductionmentioning

confidence: 99%

Towards TotalSegmentator for MRI data leveraging GIN data augmentation

Geißler,

Mensing,

Wenzel

et al. 2024

Medical Imaging 2024: Image Processing

View full text Add to dashboard Cite

Previous work on methods for cross domain generalization in medical imaging found a simple but very effective method called "global intensity non-linear" (GIN) augmentation. Our goal in this study is to use the GIN approach to train a model as powerful as TotalSegmentator for MRI data, despite having neither sufficient amounts of MRI data nor ground truth organ contours. Instead, we employ the GIN augmentation approach to show qualitatively and quantitatively that this is indeed feasible for a diverse set of anatomical structures including abdominal and thoracic organs as well as bones. The models are trained on the TotalSegmentator and AMOS22 datasets. For evaluation we apply them to whole body MRI scans from the German National Cohort (NAKO) study with a set of in-house reference masks. With GIN augmentation the mean Dice score of the model increases from 0.18 to 0.52 on Dixon water images, when using TotalSegmentator data for training. The improvements can be further split into 0.47 to 0.66 for abdominal organs, 0.55 to 0.79 for thoracic organs and 0.00 to 0.40 for bones.

show abstract

“…Ge et al [35] further integrated structural constraints into CycleGAN to overcome anatomical discrepancies through mutual information and L1 loss between original MR and synthetic CT images. Liu et al [36] refined this approach by incorporating a cost function based on the structural similarity index to generate synthetic CT images from prostate MR images. From CT to MR domain, Dong et al [37] applied CycleGAN to generate synthetic MR to improve the segmentation of multiple organs in pelvic CT images.…”

Section: Introductionmentioning

confidence: 99%

Anatomy-constrained synthesis for spleen segmentation improvement in unpaired mouse micro-CT scans with 3D CycleGAN

Jiang,

Xu,

Sheng

2024

Biomed. Phys. Eng. Express

View full text Add to dashboard Cite

Objective: Auto-segmentation in mouse micro-CT enhances the efficiency and consistency of preclinical experiments but often struggles with low-native-contrast and morphologically complex organs, such as the spleen, resulting in poor segmentation performance. While CT contrast agents can improve organ conspicuity, their use complicates experimental protocols and reduces feasibility. We developed a 3D Cycle Generative Adversarial Network (CycleGAN) incorporating anatomy-constrained U-Net models to leverage contrast-enhanced CT (CECT) insights to improve unenhanced native CT (NACT) segmentation. Approach: We employed a standard CycleGAN with an anatomical loss function to synthesize virtual CECT images from unpaired NACT scans at two different resolutions. Prior to training, two U-Nets were trained to automatically segment six major organs in NACT and CECT datasets, respectively. These pretrained 3D U-Nets were integrated during the CycleGAN training, segmenting synthetic images, and comparing them against ground truth annotations. The compound loss within the CycleGAN maintained anatomical fidelity. Full image processing was achieved for low-resolution datasets, while high-resolution datasets employed a patch-based method due to GPU memory constraints. Automated segmentation was applied to original NACT and synthetic CECT scans to evaluate CycleGAN performance using the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD95p).Main Results:High-resolution scans showed improved auto-segmentation, with an average DSC increase from 0.728 to 0.773 and a reduced HD95p from 1.19 mm to 0.94 mm. Low-resolution scans benefited more from synthetic contrast, showing a DSC increase from 0.586 to 0.682 and an HD95p reduction from 3.46 mm to 1.24 mm. Significance:Implementing CycleGAN to synthesize CECT scans substantially improved the visibility of the mouse spleen, leading to more precise auto-segmentation. This approach shows the potential in preclinical imaging studies where contrast agent use is impractical.

show abstract

Cross-Modality Knowledge Transfer for Prostate Segmentation from CT Scans

Cited by 15 publications

References 13 publications

Empowering Vision Transformer by Network Hyper-Parameter Selection for Whole Pelvis Prostate Planning Target Volume Auto-Segmentation

Empowering Vision Transformer by Network Hyper-Parameter Selection for Whole Pelvis Prostate Planning Target Volume Auto-Segmentation

Towards TotalSegmentator for MRI data leveraging GIN data augmentation

Anatomy-constrained synthesis for spleen segmentation improvement in unpaired mouse micro-CT scans with 3D CycleGAN

Contact Info

Product

Resources

About