MR to CT synthesis with multicenter data in the pelvic area using a conditional generative adversarial network

Abstract

The establishment of an MRI-only workflow in radiotherapy depends on the ability to generate an accurate synthetic CT (sCT) for dose calculation. Previously proposed methods have used a Generative Adversarial Network (GAN) for fast sCT generation in order to simplify the clinical workflow and reduces uncertainties. In the current paper we use a conditional Generative Adversarial Network (cGAN) framework called pix2pixHD to create a robust model prone to multicenter data. This study included T2-weighted MR and CT images of 19 patients in treatment position from 3 different sites. The cGAN was trained on 2D transverse slices of 11 patients from 2 different sites. Once trained, the network was used to generate sCT images of 8 patients coming from a third site. The Mean Absolute Errors (MAE) for each patient were evaluated between real and synthetic CTs. A radiotherapy plan was optimized on the sCT series and re-calculated on CTs to assess the dose distribution in terms of voxel-wise dose difference and Dose Volume Histograms (DVH) analysis. It takes on average of to generate a complete sCT (88 slices) for a patient on our GPU. The average MAE in HU between the sCT and actual patient CT (within the body contour) is 48.5 ± 6 HU with our method. The maximum dose difference to the target is 1.3%. This study demonstrates that an sCT can be generated in a multicentric context, with fewer pre-processing steps while being fast and accurate.

Pix2pix Conditional Generative Adversarial Networks for Scheimpflug Camera Color-Coded Corneal Tomography Image Generation

“…The presence of artifacts may be partially due to the low amount of data and the transposed convolutions used in the decoder part of the generator architecture. 39 Subjective assessment of synthesized images was satisfactory in agreement with objective evaluation results and other reports. 7 , 35 Also, the quality of the automated annotation algorithm was promising, giving the synthesized images a realistic appearance that challenged human readers in discrimination between original and synthesized images.…”

A review on medical imaging synthesis using deep learning and its clinical applications

“…Boni et al . recently presented a proof‐of‐concept study that predicted synthetic images of one clinical site using a model trained on data from two other sites and demonstrated clinically acceptable results 142 . Further studies could include datasets from multiple centers and adopt a leave‐one‐center‐out training and/or test strategy in order to validate the consistency and robustness of the network.…”

Prediction of the Short-Term Therapeutic Effect of Anti-VEGF Therapy for Diabetic Macular Edema Using a Generative Adversarial Network with OCT Images

“…Among existing GAN algorithms, the pix2pixHD algorithm is well-established as outperforming other competitive algorithms in many cases and was selected as our main algorithm in the training process. In addition, the comparison of pix2pixHD against other state-of-the-art algorithms such as pix2pix and CRN was included [ 13 , 14 , 15 ]. Roughly speaking, the pix2pixHD algorithm yields a high-resolution image-generating neural network in a game-playing manner.…”