Unsupervised Medical Image Translation Using Cycle-MedGAN

Armanious, Karim; Jiang, Chenming; Abdulatif, Sherif; Küstner, Thomas; Gatidis, Sergios; Yang, Bin

doi:10.23919/eusipco.2019.8902799

Cited by 92 publications

(72 citation statements)

References 26 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The contrast between bone and soft tissue is significantly larger than the contrast between fat and muscle. In previous cycle-GAN methods proposed for medical imaging, where the loss function has been intensity based, for example, mean absolute error (MAE) used by Armanious et al, 39 or structure based, for example, gradient difference (GD) used in our previous work, 36 it cannot be guaranteed that the trained cycle-GAN model will be able to differentiate tissues with such similar intensities. Thus, in this study, a statistical matching-based loss was integrated into generator's loss function.…”

Section: C Cycle-gan For Rsp Image Generationmentioning

confidence: 94%

Cone‐beam CT‐derived relative stopping power map generation via deep learning for proton radiotherapy

et al. 2020

View full text Add to dashboard Cite

In intensity-modulated proton therapy (IMPT), protons are used to deliver highly conformal dose distributions, targeting tumors, and sparing organs-at-risk. However, due to uncertainties in both patient setup and relative stopping power (RSP) calculation, margins are added to the treatment volume during treatment planning, leading to higher doses to normal tissues. Cone-beam computed tomography (CBCT) images are taken daily before treatment; however, the poor image quality of CBCT limits the use of these images for online dose calculation. In this work, we use a deep-learning-based method to predict RSP maps from daily CBCT images, allowing for online dose calculation in a step toward adaptive radiation therapy. Methods: Twenty-three head-and-neck cancer patients were simulated using a Siemens TwinBeam dual-energy CT (DECT) scanner. Mixed-energy scans (equivalent to a 120 kVp single-energy CT scan) were converted to RSP maps for treatment planning. Cone-beam computed tomography images were taken on the first day of treatment, and the planning RSP maps were registered to these images. A deep learning network based on a cycle-GAN architecture, relying on a compound loss function designed for structural and contrast preservation, was then trained to create an RSP map from a CBCT image. Leave-one-out and holdout cross validations were used for evaluation, and mean absolute error (MAE), mean error (ME), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) were used to quantify the differences between the CT-based and CBCT-based RSP maps. The proposed method was compared to a deformable image registration-based method which was taken as the ground truth and two other deep learning methods. For one patient who underwent resimulation, the new planning RSP maps and CBCT images were used for further evaluation and validation. Results: The CBCT-based RSP generation method was evaluated on 23 head-and-neck cancer patients. From leave-one-out testing, the MAE between CT-based and CBCT-based RSP was 0.06 AE 0.01 and the ME was −0.01 AE 0.01. The proposed method statistically outperformed the comparison DL methods in terms of MAE and ME when compared to the planning CT. In terms of dose comparison, the mean gamma passing rate at 3%/3 mm was 94% when three-dimensional (3D) gamma index was calculated per plan and 96% when gamma index was calculated per field. Conclusions: The proposed method provides sufficiently accurate RSP map generation from CBCT images, allowing for evaluation of daily dose based on CBCT and possibly allowing for CBCTguided adaptive treatment planning for IMPT.

show abstract

Section: C Cycle-gan For Rsp Image Generationmentioning

confidence: 94%

Cone‐beam CT‐derived relative stopping power map generation via deep learning for proton radiotherapy

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The discriminator, which is a typical CNN, attempts to distinguish fake data generated by the generative network from real data. Variations of GANs, such as cycleGAN, are very powerful tools in unsupervised image translation, such as MRI to CT conversion (Armanious et al 2019 ).…”

Section: Principles Of Machine Learning and Deep Learningmentioning

confidence: 99%

Applications of artificial intelligence and deep learning in molecular imaging and radiotherapy

Arabi

Zaidi

2020

European J Hybrid Imaging

128

View full text Add to dashboard Cite

This brief review summarizes the major applications of artificial intelligence (AI), in particular deep learning approaches, in molecular imaging and radiation therapy research. To this end, the applications of artificial intelligence in five generic fields of molecular imaging and radiation therapy, including PET instrumentation design, PET image reconstruction quantification and segmentation, image denoising (low-dose imaging), radiation dosimetry and computer-aided diagnosis, and outcome prediction are discussed. This review sets out to cover briefly the fundamental concepts of AI and deep learning followed by a presentation of seminal achievements and the challenges facing their adoption in clinical setting.

show abstract

“…Recently, Kearney et al [ 40 ] involved the attention mechanism in CCGANs to perform image translation of MRI to CT scans. Similarly, Armanious et al [ 41 ] translated the Positron Emission-computed Tomography (PET) images to CT scan images using CCGANs along with nonadversarial cycle losses. On the other hand, when it comes to gait recognition, Yu et al [ 42 ] proposed a framework, namely, GaitGAN to select invariant features of an individual's gait to reduce the effect of covariate factors.…”

Section: Literature Reviewmentioning

confidence: 99%

A Smart Surveillance System for Uncooperative Gait Recognition Using Cycle Consistent Generative Adversarial Networks (CCGANs)

Alsaggaf

Mehmood

Khairullah

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Surveillance remains an important research area, and it has many applications. Smart surveillance requires a high level of accuracy even when persons are uncooperative. Gait Recognition is the study of recognizing people by the way they walk even when they are unwilling to cooperate. It is another form of a behavioral biometric system in which unique attributes of an individual’s gait are analyzed to determine their identity. On the other hand, one of the big limitations of the gait recognition system is uncooperative environments in which both gallery and probe sets are made under different and unknown walking conditions. In order to tackle this problem, we propose a deep learning-based method that is trained on individuals with the normal walking condition, and to deal with an uncooperative environment and recognize the individual with any dynamic walking conditions, a cycle consistent generative adversarial network is used. This method translates a GEI disturbed from different covariate factors to a normal GEI. It works like unsupervised learning, and during its training, a GEI disrupts from different covariate factors of each individual and acts as a source domain while the normal walking conditions of individuals are our target domain to which translation is required. The cycle consistent GANs automatically find an individual pair with the help of the Cycle Loss function and generate the required GEI, which is tested by the CNN model to predict the person ID. The proposed system is evaluated over a publicly available data set named CASIA-B, and it achieved excellent results. Moreover, this system can be implemented in sensitive areas, like banks, seminar halls (events), airports, embassies, shopping malls, police stations, military areas, and other public service areas for security purposes.

show abstract

Unsupervised Medical Image Translation Using Cycle-MedGAN

Cited by 92 publications

References 26 publications

Cone‐beam CT‐derived relative stopping power map generation via deep learning for proton radiotherapy

Cone‐beam CT‐derived relative stopping power map generation via deep learning for proton radiotherapy

Applications of artificial intelligence and deep learning in molecular imaging and radiotherapy

A Smart Surveillance System for Uncooperative Gait Recognition Using Cycle Consistent Generative Adversarial Networks (CCGANs)

Contact Info

Product

Resources

About