Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images

Uzunova, Hristina; Ehrhardt, Jan; Jacob, F.; Frydrychowicz, Alex; Handels, Heinz

doi:10.1007/978-3-030-32226-7_13

Cited by 30 publications

(11 citation statements)

References 8 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To generate high resolution 2D and 3D images, a multiscale GAN with patch-wise learning was developed (48). Starting from a low-resolution scale of the image, the training was repeated and conditioned on the previous scale to produce a higher resolution.…”

Section: Srmentioning

confidence: 99%

Narrative review of generative adversarial networks in medical and molecular imaging

Koshino¹,

Werner²,

Pomper³

et al. 2021

Ann Transl Med

View full text Add to dashboard Cite

Recent years have witnessed a rapidly expanding use of artificial intelligence and machine learning in medical imaging. Generative adversarial networks (GANs) are techniques to synthesize images based on artificial neural networks and deep learning. In addition to the flexibility and versatility inherent in deep learning on which the GANs are based, the potential problem-solving ability of the GANs has attracted attention and is being vigorously studied in the medical and molecular imaging fields. Here this narrative review provides a comprehensive overview for GANs and discuss their usefulness in medical and molecular imaging on the following topics: (I) data augmentation to increase training data for AI-based computeraided diagnosis as a solution for the data-hungry nature of such training sets; (II) modality conversion to complement the shortcomings of a single modality that reflects certain physical measurement principles, such as from magnetic resonance (MR) to computed tomography (CT) images or vice versa; (III) de-noising to realize less injection and/or radiation dose for nuclear medicine and CT; (IV) image reconstruction for shortening MR acquisition time while maintaining high image quality; (V) super-resolution to produce a high-resolution image from low-resolution one; (VI) domain adaptation which utilizes knowledge such as supervised labels and annotations from a source domain to the target domain with no or insufficient knowledge; and (VII) image generation with disease severity and radiogenomics. GANs are promising tools for medical and molecular imaging. The progress of model architectures and their applications should continue to be noteworthy.

show abstract

Section: Srmentioning

confidence: 99%

Narrative review of generative adversarial networks in medical and molecular imaging

Koshino¹,

Werner²,

Pomper³

et al. 2021

Ann Transl Med

View full text Add to dashboard Cite

show abstract

“…It has not been proven so far that low FID scores induce high image quality when applied to medical images. However, recent works indicate correlation between FID score and realism of generated medical images [13,21] (5)…”

Section: Fréchet Inception Distancementioning

confidence: 99%

SpeckleGAN: a generative adversarial network with an adaptive speckle layer to augment limited training data for ultrasound image processing

Bargsten

Schlaefer

2020

Int J CARS

View full text Add to dashboard Cite

Purpose In the field of medical image analysis, deep learning methods gained huge attention over the last years. This can be explained by their often improved performance compared to classic explicit algorithms. In order to work well, they need large amounts of annotated data for supervised learning, but these are often not available in the case of medical image data. One way to overcome this limitation is to generate synthetic training data, e.g., by performing simulations to artificially augment the dataset. However, simulations require domain knowledge and are limited by the complexity of the underlying physical model. Another method to perform data augmentation is the generation of images by means of neural networks. Methods We developed a new algorithm for generation of synthetic medical images exhibiting speckle noise via generative adversarial networks (GANs). Key ingredient is a speckle layer, which can be incorporated into a neural network in order to add realistic and domain-dependent speckle. We call the resulting GAN architecture SpeckleGAN. Results We compared our new approach to an equivalent GAN without speckle layer. SpeckleGAN was able to generate ultrasound images with very crisp speckle patterns in contrast to the baseline GAN, even for small datasets of 50 images. SpeckleGAN outperformed the baseline GAN by up to 165 % with respect to the Fréchet Inception distance. For artery layer and lumen segmentation, a performance improvement of up to 4 % was obtained for small datasets, when these were augmented with images by SpeckleGAN. Conclusion SpeckleGAN facilitates the generation of realistic synthetic ultrasound images to augment small training sets for deep learning based image processing. Its application is not restricted to ultrasound images but could be used for every imaging methodology that produces images with speckle such as optical coherence tomography or radar.

show abstract

“…Since the GAN concept was first published in 2014, there have been numerous advances in training methods, including the integration of deep convolutional architectures [10], and the development of the progressive GAN training method [11]. Prior medical applications of GANs have included the synthesis of pathologic images of breast cancer [12], gliomas [13], and cervical dysplasia [14], as well as images of macular degeneration [15], dermatologic conditions [16], and several types of radiographic modality [17–19]. However, the images generated in these previous works were generally quite limited in size and had a low resolution.…”

Section: Introductionmentioning

confidence: 99%

Synthesis of diagnostic quality cancer pathology images by generative adversarial networks

Levine

Peng

Farnell

et al. 2020

The Journal of Pathology

View full text Add to dashboard Cite

Deep learning‐based computer vision methods have recently made remarkable breakthroughs in the analysis and classification of cancer pathology images. However, there has been relatively little investigation of the utility of deep neural networks to synthesize medical images. In this study, we evaluated the efficacy of generative adversarial networks to synthesize high‐resolution pathology images of 10 histological types of cancer, including five cancer types from The Cancer Genome Atlas and the five major histological subtypes of ovarian carcinoma. The quality of these images was assessed using a comprehensive survey of board‐certified pathologists (n = 9) and pathology trainees (n = 6). Our results show that the real and synthetic images are classified by histotype with comparable accuracies and the synthetic images are visually indistinguishable from real images. Furthermore, we trained deep convolutional neural networks to diagnose the different cancer types and determined that the synthetic images perform as well as additional real images when used to supplement a small training set. These findings have important applications in proficiency testing of medical practitioners and quality assurance in clinical laboratories. Furthermore, training of computer‐aided diagnostic systems can benefit from synthetic images where labeled datasets are limited (e.g. rare cancers). We have created a publicly available website where clinicians and researchers can attempt questions from the image survey (http://gan.aimlab.ca/). © 2020 Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

show abstract

Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images

Cited by 30 publications

References 8 publications

Narrative review of generative adversarial networks in medical and molecular imaging

Narrative review of generative adversarial networks in medical and molecular imaging

SpeckleGAN: a generative adversarial network with an adaptive speckle layer to augment limited training data for ultrasound image processing

Synthesis of diagnostic quality cancer pathology images by generative adversarial networks

Contact Info

Product

Resources

About