Investigating conditional GAN performance with different generator architectures, an ensemble model, and different MR scanners for MR-sCT conversion

Abstract

Recent developments in magnetic resonance (MR) to synthetic computed tomography (sCT) conversion have shown that treatment planning is possible without an initial planning CT. Promising conversion results have been demonstrated recently using conditional generative adversarial networks (cGANs). However, the performance is generally only tested on images from one MR scanner, which neglects the potential of neural networks to find general high-level abstract features. In this study, we explored the generalizability of the generator models, trained on a single field strength scanner, to data acquired with higher field strengths. T2-weighted 0.35T MRIs and CTs from 51 patients treated for prostate (40) and cervical cancer (11) were included. 25 of them were used to train four different generators (SE-ResNet, DenseNet, U-Net, and Embedded Net). Further, an ensemble model was created from the four network outputs. The models were validated on 16 patients from a 0.35T MR scanner. Further, the trained models were tested on the Gold Atlas dataset, containing T2-weighted MR scans of different field strengths; 1.5T(7) and 3T(12), and 10 patients from the 0.35T scanner. The sCTs were dosimetrically compared using clinical VMAT plans for all test patients. For the same scanner (0.35T), the results from the different models were comparable on the test set, with only minor differences in the mean absolute error (MAE) (35-51HU body). Similar results were obtained for conversions of 3T GE Signa and the 3T GE Discovery images (40-62HU MAE) for three of the models. However, larger differences were observed for the 1.5T images (48-65HU MAE). The overall best model was found to be the ensemble model. All dose differences were below 1%. This study shows that it is possible to generalize models trained on images of one scanner to other scanners and different field strengths. The best metric results were achieved by the combination of all networks.

Deep learning based synthetic‐CT generation in radiotherapy and PET: A review

“…To date, four scientific studies have already investigated the performance of DL‐based sCT in a multicenter setting 71,114–116 . These studies have been reported only for MR‐only RT.…”

“…A detailed examination of different 2D-paired GANs against U-net with different loss functions by Largent et al 113 showed that U-net and GANs could achieve similar image-and dose-base performances. Fetty et al 115 focused on comparing different generators of a 2D-paired GAN against the performance of an ensemble of models, finding that the ensemble was overall better than single models being more robust to generalization on data from different scanners/centers. When considering CNNs architectures, it is worth mentioning using 2.5D-dilated CNNs by Dinkla et al, 102 where the m2D training was claimed to increase the robustness of inference in a 2D+ manner, maintaining a big receptive field and a low number of weights.…”

Synthetic CT Generation of the Pelvis in Patients With Cervical Cancer: A Single Input Approach Using Generative Adversarial Network

“…This being said, the relationship between the amplitude of the sCT error and the subsequent error inaccuracies in PET attenuation correction and radiation therapy dose planning is not predictable. This was previously studied by Fetty et al who found no correlation between MAPE and the radiation therapy dosimetric plan evaluation [ 35 ]. Under this perspective, we focused the validation of our study on HU accuracy and the convenience of our method in the context of the usual diagnostic and therapeutic workflow in gynecological malignancy.…”

“…As for the hyperparameter λ , an initial analysis of its effect had already been initiated by Isola et al . [ 31 ], and was subsequently adopted by different researchers in the medical imaging field [ 35 ], [ 37 ], [ 38 ]in which the choice of λ = 100 yielded the best results. In line with the previously published literature [ 31 ], [ 37 ], [ 38 ], we chose λ = 100 in the U-cGAN and sU-cGAN loss functions.…”

“…Using a similar approach, Brou Boni et a l. reported sCT generation from T2-weighted MR images in 19 male patients with prostate or rectal cancer [ 34 ]. In addition, Fetty et al also used T2-weighted MR images to compare the performance of different cGAN generators across different MR magnetic field strengths [ 35 ]. To the best of our knowledge, cGAN-based pelvis sCT generation via T2-weighted TSE-SSH has not yet been studied.…”