Evaluating the Performance of StyleGAN2-ADA on Medical Images

Abstract

Although generative adversarial networks (GANs) have shown promise in medical imaging, they have four main limitations that impede their utility: computational cost, data requirements, reliable evaluation measures, and training complexity. Our work investigates each of these obstacles in a novel application of StyleGAN2-ADA to high-resolution medical imaging datasets. Our dataset is comprised of liver-containing axial slices from non-contrast and contrast-enhanced computed tomography (CT) scans. Additionally, we utilized four public datasets composed of various imaging modalities. We trained a StyleGAN2 network with transfer learning (from the Flickr-Faces-HQ dataset) and data augmentation (horizontal flipping and adaptive discriminator augmentation). The network's generative quality was measured quantitatively with the Fréchet Inception Distance (FID) and qualitatively with a visual Turing test given to seven radiologists and radiation oncologists.The StyleGAN2-ADA network achieved a FID of 5.22 (± 0.17) on our liver CT dataset. It also set new record FIDs of 10.78, 3.52, 21.17, and 5.39 on the publicly available SLIVER07, ChestX-ray14, ACDC, and Medical Segmentation Decathlon (brain tumors) datasets. In the visual Turing test, the clinicians rated generated images as real 42% of the time, approaching random guessing. Our computational ablation study revealed that transfer learning and data augmentation stabilize training and improve the perceptual quality of the generated images. We observed the FID to be consistent with human perceptual evaluation of medical images. Finally, our work found that StyleGAN2-ADA consistently produces high-quality results without hyperparameter searches or retraining.

Evaluating the performance of generative adversarial network-synthesized periapical images in classifying C-shaped root canals

“…Thus, some studies argue that using FID for medical imaging is neither practical nor feasible and suggest replacing the inception network with their own encoding networks 46,47 . Nonetheless, recent studies using StyleGAN2 have reported their results using FID 21,45 , which is different from the approach of using their own encoding networks for FID evaluation in medical imaging. This is because the alternative approach lacks consistency in evaluating and comparing FID because it does not use the same encoding model as ImageNet 21,48 .…”

“…These results may appear unsatisfactory when compared with other medical studies. One study 21 reported FID scores of 5.22 (± 0.17) for a liver CT dataset on a StyleGAN2 network with transfer learning from the FFHQ dataset, and FIDs of 10.78, 3.52, 21.17, and 5.39 on the publicly available SLIVER07, ChestX-ray14, ACDC, and Medical Segmentation Decathlon (brain tumors) datasets. In another study 45 , the FID was approximately 20 for synthesized magnetic resonance and CT images.…”

Generative Modeling of the Circle of Willis Using 3D-StyleGAN

“…The FID has been reported to be over-reliant on texture and is argued not to be directly transferable to medical images (Hong et al, 2021). Other works argue that FID aligns well with visual quality analyses (Woodland et al, 2022). In addition to FID, we therefore use the MedicalNet (Chen et al, 2019) for feature extraction before calculating the Fréchet Distance.…”

Evaluating Synthetic Medical Images Using Artificial Intelligence with the GAN Algorithm

“…In addition to the above automated methods, evaluation approaches, which involve humans/experts, have also been used, for example, the visual turing test [ 41 ], five-point Likert scale [ 38 ], and human eye perceptual evaluation (HYPE) [ 39 ]. Although these methods are considered the most accurate methods and are the gold standard, they are costly and time-consuming.…”