ScarGAN: Chained Generative Adversarial Networks to Simulate Pathological Tissue on Cardiovascular MR Scans

Abstract

Medical images with specific pathologies are scarce, but a large amount of data is usually required for a deep convolutional neural network (DCNN) to achieve good accuracy. We consider the problem of segmenting the left ventricular (LV) myocardium on late gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) scans of which only some of the scans have scar tissue. We propose ScarGAN to simulate scar tissue on healthy myocardium using chained generative adversarial networks (GAN). Our novel approach factorizes the simulation process into 3 steps: 1) a mask generator to simulate the shape of the scar tissue; 2) a domain-specific heuristic to produce the initial simulated scar tissue from the simulated shape; 3) a refining generator to add details to the simulated scar tissue. Unlike other approaches that generate samples from scratch, we simulate scar tissue on normal scans resulting in highly realistic samples. We show that experienced radiologists are unable to distinguish between real and simulated scar tissue. Training a U-Net with additional scans with scar tissue simulated by ScarGAN increases the percentage of scar pixels correctly included in LV myocardium prediction from 75.9% to 80.5%.

CT-To-MR Conditional Generative Adversarial Networks for Ischemic Stroke Lesion Segmentation

“…A future direction of this work is to synthesize additional training examples by building generative models [11], [12] to generate new examples that can capture the large variability in shape, appearance, and location of pathological tissue. This has the advantage of being able to bolster limited training datasets by synthesizing artificial examples.…”

“…Here, GANs were used to augment healthy MR scans with realistic-looking scar tissue. Using two GANs, [12] were able to both generate the scar tissue and refine intensity values using a domain-specific heuristic. The generation of scar tissue was effective in training and led to realistic resultsexperienced physicians mistook the generated images as being real.…”

Diagnostic utility of artificial intelligence for left ventricular scar identification using cardiac magnetic resonance imaging—A systematic review

“…Metadata can vary between manufacturers and magnet strengths 22 and models trained with specific data may not be generally applicable. Only a small subset of studies included multiple CMR manufactures to mitigate this risk 14,21,[23][24][25][26] . Training data on various field strengths would allow greater generalisability clinically but 1.5T remains the current standard with 3T employed mainly by more experienced imaging centres 5 .…”

“…Other smaller datasets exist in the form of CMR challenges for more standardised model comparison. Further options to improve generalisability within existing datasets include augmentation through image transformations or using generative adversarial networks to produce synthetic images 23,25 .…”

A primer in artificial intelligence in cardiovascular medicine

“…In rare diseases, pathology can even be simulated to create artificial training data [17]. Interestingly, DL models are sometimes considered to be ‘black boxes’ and, in general, provide little insight into what they learn and how it was learnt.…”