Improving CBCT quality to CT level using deep learning with generative adversarial network

Abstract

Purpose To improve image quality and computed tomography (CT) number accuracy of daily cone beam CT (CBCT) through a deep learning methodology with generative adversarial network. Methods One hundred and fifty paired pelvic CT and CBCT scans were used for model training and validation. An unsupervised deep learning method, 2.5D pixel‐to‐pixel generative adversarial network (GAN) model with feature mapping was proposed. A total of 12 000 slice pairs of CT and CBCT were used for model training, while ten‐fold cross validation was applied to verify model robustness. Paired CT–CBCT scans from an additional 15 pelvic patients and 10 head‐and‐neck (HN) patients with CBCT images collected at a different machine were used for independent testing purpose. Besides the proposed method above, other network architectures were also tested as: 2D vs 2.5D; GAN model with vs without feature mapping; GAN model with vs without additional perceptual loss; and previously reported models as U‐net and cycleGAN with or without identity loss. Image quality of deep‐learning generated synthetic CT (sCT) images was quantitatively compared against the reference CT (rCT) image using mean absolute error (MAE) of Hounsfield units (HU) and peak signal‐to‐noise ratio (PSNR). The dosimetric calculation accuracy was further evaluated with both photon and proton beams. Results The deep‐learning generated sCTs showed improved image quality with reduced artifact distortion and improved soft tissue contrast. The proposed algorithm of 2.5 Pix2pix GAN with feature matching (FM) was shown to be the best model among all tested methods producing the highest PSNR and the lowest MAE to rCT. The dose distribution demonstrated a high accuracy in the scope of photon‐based planning, yet more work is needed for proton‐based treatment. Once the model was trained, it took 11–12 ms to process one slice, and could generate a 3D volume of dCBCT (80 slices) in less than a second using a NVIDIA GeForce GTX Titan X GPU (12 GB, Maxwell architecture). Conclusion The proposed deep learning algorithm is promising to improve CBCT image quality in an efficient way, thus has a potential to support online CBCT‐based adaptive radiotherapy.

Deep learning based synthetic‐CT generation in radiotherapy and PET: A review

“…83,[124][125][126][127] The remaining 10 proved the validity of the transformation with dosimetric studies for photons, 66,71,101,[128][129][130][131] protons, 120 and for both photons and protons. 85,132,133 Only three studies investigated unpaired training 84,128,133 ; in 11 cases, paired training was implemented by matching the CBCT and ground truth CT by rigid or deformable registration. In Eck et al, 66 however, CBCT and CT were not registered for the training phase, as the authors claimed the first fraction CBCT was geometrically close enough to the planning CT for the network.…”

A Comparative Study on the Voxel Values in Alveolar Bones Acquired by MDCT and Newly Developed Dental Dual-Energy CBCT

“…RCT720 took approximately 30 s from the start of the patient scan to image reconstruction and provided resultant images. This was expected to be faster than deep learning techniques [20,21], which perform massive computation through complex neural networks after patient scan and image reconstruction. Furthermore, we expect that it would be more convenient and familiar to assess bone quality with the voxel values of CBCT images than with bone density values [21].…”

“…These physical limitations result in inaccurate voxel values and bone density assessments [18]. In the field of radiation therapy, some attempts to utilize deep learning techniques, such as the generative adversarial network (GAN), to correct CBCT artifacts, and to improve image quality of CBCT images were made [19,20]. A feasibility study was conducted to measure bone density directly and quantitatively from CBCT images using modified GAN [21].…”

Simple Code Implementation for Deep Learning–Based Segmentation to Evaluate Central Serous Chorioretinopathy in Fundus Photography

“… 11 , 24 , 25 A previous study using pelvic computed tomography images showed that pix2pix outperformed U-Net and other GAN techniques. 12 In this manner, we performed an experiment using U-Net based on pix2pix for pathological lesion segmentation in fundus photography.…”

“… 10 Because the single U-Net model does not consider the detailed features of the output images, generative adversarial network (GAN) framework can improve the performance of the U-Net model. 11 , 12 In GAN architecture, the generator and discriminator operate as adversaries to synthesize more realistic output images. 13 Pix2pix is a popular GAN technique using U-Net for image-to-image translation.…”