Evaluating the Clinical Realism of Synthetic Chest X-Rays Generated Using Progressively Growing GANs

Abstract

Chest X-rays are a vital diagnostic tool in the workup of many patients. Similar to most medical imaging modalities, they are profoundly multi-modal and are capable of visualising a variety of combinations of conditions. There is an ever pressing need for greater quantities of labelled images to drive forward the development of diagnostic tools; however, this is in direct opposition to concerns regarding patient confidentiality which constrains access through permission requests and ethics approvals. Previous work has sought to address these concerns by creating class-specific generative adversarial networks (GANs) that synthesise images to augment training data. These approaches cannot be scaled as they introduce computational trade offs between model size and class number which places fixed limits on the quality that such generates can achieve. We address this concern by introducing latent class optimisation which enables efficient, multi-modal sampling from a GAN and with which we synthesise a large archive of labelled generates. We apply a Progressive Growing GAN (PGGAN) to the task of unsupervised X-ray synthesis and have radiologists evaluate the clinical realism of the resultant samples. We provide an in depth review of the properties of varying pathologies seen on generates as well as an overview of the extent of disease diversity captured by the model. We validate the application of the Fréchet Inception Distance (FID) to measure the quality of X-ray generates and find that they are similar to other high-resolution tasks. We quantify X-ray clinical realism by asking radiologists to distinguish between real and fake scans and find that generates are more likely to be classed as real than by chance, but there is still progress required to achieve true realism. We confirm these findings by evaluating synthetic classification model performance on real scans. We conclude by discussing the limitations of PGGAN generates and how to achieve controllable, realistic generates going forward. We release our source code, model weights, and an archive of labelled generates.

Evaluating clinical diversity and plausibility of synthetic capsule endoscopic images

“…Recent deep learning-based generation methods in WCE although reasonably realistic [17,23], have little or no control over the clinical attributes comprising each generated image (hereby referred to as generates). Methods that enable the control of meaningful clinical markers of disease so that realistic images with desirable attributes can be simulated on cue is a fairly new research direction [24][25][26][27] that this work delves into. Moreover, the evaluation of synthetic datasets using pretrained object detectors as in [23] while useful in assessing realness to some extent, does not reliably measure the diversity within the dataset.…”

“…Such controlled generation has been explored in modalities like retinal, CT, X-ray, etc. [24,25,32]. However, the aforementioned medical domains are simpler than WCE.…”

Evaluating the Performance of StyleGAN2-ADA on Medical Images

“…The ChestX-ray14 dataset 6 [28] consists of 112,120 1024x1024 Chest X-ray images in PNG format. The previous best FID on the ChestX-ray14 dataset of 8.02 was achieved using a Progressive Growing GAN [24]. No preprocessing on this dataset was performed.…”

“…The original StyleGAN2 project took 51.06 GPU years to create, 0.23 of which were used for training the Flickr-Faces-HQ (FFHQ) weights used in our paper [15]. Despite being the state-ofthe-art generative model for high-resolution images, StyleGAN2 is often not used in medical imaging literature due to its expense [24]. If it is used, images are brought to lower resolutions to offset the cost [20,22].…”

Generation of microbial colonies dataset with deep learning style transfer

“…GANs are used for generation of high quality radiology images, e.g. mammograms for radiology education 27 , or realistic chest X-ray images 28 . In dermatology, for automated skin cancer classification 29 , or to build a general skin condition classifier 30 .…”