Generating Synthetic Labeled Data From Existing Anatomical Models: An Example With Echocardiography Segmentation

Abstract

Deep learning can bring time savings and increased reproducibility to medical image analysis. However, acquiring training data is challenging due to the time-intensive nature of labeling and high inter-observer variability in annotations. Rather than labeling images, in this work we propose an alternative pipeline where images are generated from existing high-quality annotations using generative adversarial networks (GANs). Annotations are derived automatically from previously built anatomical models and are transformed into realistic synthetic ultrasound images with paired labels using a CycleGAN. We demonstrate the pipeline by generating synthetic 2D echocardiography images to compare with existing deep learning ultrasound segmentation datasets. A convolutional neural network is trained to segment the left ventricle and left atrium using only synthetic images. Networks trained with synthetic images were extensively tested on four different unseen datasets of real images with median Dice scores of 91, 90, 88, and 87 for left ventricle segmentation. These results match or are better than inter-observer results measured on real ultrasound datasets and are comparable to a network trained on a separate set of real images. Results demonstrate the images produced can effectively be used in place of real data for training. The proposed pipeline opens the door for automatic generation of training data for many tasks in medical imaging as the same process can be applied to other segmentation or landmark detection tasks in any modality. The source code and anatomical models are available to other researchers. 1 1

Clinically-Driven Virtual Patient Cohorts Generation: An Application to Aorta

“…Other authors do focus in shape generation, mainly with medical image as the source of information. Gilbert et al (2021) generate synthetic 2D, labeled echocardiography images using GAN, and then train a convolutional neural network segment the left ventricle and left atrium using only synthetic images. Rodero et al (2021) link the main deformations of a cohort of 19 healthy hearts with the electrophysiological biomarkers acquired via simulation.…”

“…The segmentation of clinical images is time-consuming and suffers from observer variability. Despite the promising results in the automation of the process via machine learning approaches (Bratt et al, 2019 ; Hepp et al, 2020 ; Gilbert et al, 2021 ), these models fail to generalize well if the object of interest is infrequent in the population. Thus, in clinical practice, image processing and segmentation remains mostly a semi-automatic task.…”

A Data Augmentation Pipeline to Generate Synthetic Labeled Datasets of 3D Echocardiography Images Using a GAN

“…Huo et al [20] trained a 2D GAN model, SynSegNet, on CT images and unpaired MR labels using a CycleGAN. Similarly, [12] proposed an approach to synthesize labeled 2D echocardiography images, using anatomical models and a CycleGAN as well. The CycleGAN was proposed by [19] and works under an unpaired scenario: the images from one training domain do not have to be related with the images belonging to the other domain.…”

“…Different types of models can be used for this purpose, such as animated models, biophysical models, or even anatomical models obtained from different imaging modalities [10], [11]. Recently, CT models were used as label sources to generate 2D echocardiography [12] and cardiac MR images [13], proving the utility of GANs for this task.…”

Light-weight spatio-temporal graphs for segmentation and ejection fraction prediction in cardiac ultrasound

“…Several previous works have shown highly accurate automation results using deep learning for both left ventricle segmentation [15,11,20,9,13,4] and direct EF estimation [15,10,17]. For segmentation, previous works have mostly relied on semantic segmentation, which outputs a pixel-wise classification of an input image.…”