Unpaired PET/CT image synthesis of liver region using CycleGAN

Santini, Gianmarco; Fourcade, Constance; Moreau, Noémie; Rousseau, Caroline; Ferrer, Ludovic; Lacombe, M.; Campone, Mario; Jézéquel, Pascal; Rubeaux, Mathieu

doi:10.1117/12.2576095

Cited by 2 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, GAN-based methods were demonstrated to offer state-of-theart performance in numerous synthesis tasks, including data augmentation as well as multi-modal synthesis [24], [34], [71], [72]. Important applications of GAN models include CT to PET [73], [74], MR to CT [75]- [77], unpaired crossmodality [78]- [81], 3T-to-7T [82], [83], and multi-contrast MRI synthesis [24], [30]- [42].…”

Section: Related Workmentioning

confidence: 99%

ResViT: Residual Vision Transformers for Multimodal Medical Image Synthesis

Dalmaz

Yurt

Çukur

2022

IEEE Trans. Med. Imaging

160

View full text Add to dashboard Cite

Generative adversarial models with convolutional neural network (CNN) backbones have recently been established as state-of-the-art in numerous medical image synthesis tasks. However, CNNs are designed to perform local processing with compact filters, and this inductive bias compromises learning of contextual features. Here, we propose a novel generative adversarial approach for medical image synthesis, ResViT, that leverages the contextual sensitivity of vision transformers along with the precision of convolution operators and realism of adversarial learning. ResViT's generator employs a central bottleneck comprising novel aggregated residual transformer (ART) blocks that synergistically combine residual convolutional and transformer modules. Residual connections in ART blocks promote diversity in captured representations, while a channel compression module distills task-relevant information. A weight sharing strategy is introduced among ART blocks to mitigate computational burden. A unified implementation is introduced to avoid the need to rebuild separate synthesis models for varying source-target modality configurations. Comprehensive demonstrations are performed for synthesizing missing sequences in multi-contrast MRI, and CT images from MRI. Our results indicate superiority of ResViT against competing CNN-and transformer-based methods in terms of qualitative observations and quantitative metrics.

show abstract