Synthesizing Diverse Lung Nodules Wherever Massively: 3D Multi-Conditional GAN-Based CT Image Augmentation for Object Detection

Han, Changhee; Kitamura, Yoshiro; Kudo, Akira; Ichinose, Akimichi; Rundo, Leonardo; Furukawa, Yukio; Umemoto, Kazuki; Li, Yuanzhong; Nakayama, Hideki

doi:10.1109/3dv.2019.00085

Cited by 106 publications

(62 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The amount of homogeneous and well-prepared datasets represents an important challenge in biomedical imaging [24]. As a matter of fact, Deep Learning research has been recently focusing on issues related to medical imaging datasets with limited sample size, achieving promising performance by means of weakly-/semi-supervised schemes [49,71] as well as Generative Adversarial Network (GAN)-based data augmentation [72,73]. Moreover, methods tailored to each particular clinical application should be devised, such as for improving the model generalization abilities even in the case of small datasets collected from multiple institutions [15].…”

Section: Discussionmentioning

confidence: 99%

A Hybrid End-to-End Approach Integrating Conditional Random Fields into CNNs for Prostate Cancer Detection on MRI

et al. 2020

Self Cite

View full text Add to dashboard Cite

Prostate Cancer (PCa) is the most common oncological disease in Western men. Even though a growing effort has been carried out by the scientific community in recent years, accurate and reliable automated PCa detection methods on multiparametric Magnetic Resonance Imaging (mpMRI) are still a compelling issue. In this work, a Deep Neural Network architecture is developed for the task of classifying clinically significant PCa on non-contrast-enhanced MR images. In particular, we propose the use of Conditional Random Fields as a Recurrent Neural Network (CRF-RNN) to enhance the classification performance of XmasNet, a Convolutional Neural Network (CNN) architecture specifically tailored to the PROSTATEx17 Challenge. The devised approach builds a hybrid end-to-end trainable network, CRF-XmasNet, composed of an initial CNN component performing feature extraction and a CRF-based probabilistic graphical model component for structured prediction, without the need for two separate training procedures. Experimental results show the suitability of this method in terms of classification accuracy and training time, even though the high-variability of the observed results must be reduced before transferring the resulting architecture to a clinical environment. Interestingly, the use of CRFs as a separate postprocessing method achieves significantly lower performance with respect to the proposed hybrid end-to-end approach. The proposed hybrid end-to-end CRF-RNN approach yields excellent peak performance for all the CNN architectures taken into account, but it shows a high-variability, thus requiring future investigation on the integration of CRFs into a CNN.

show abstract

Section: Discussionmentioning

confidence: 99%

A Hybrid End-to-End Approach Integrating Conditional Random Fields into CNNs for Prostate Cancer Detection on MRI

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Schlegl et al [136] presented an approach called fast AnoGAN (f-AnoGAN), which can identify anomalous images on a variety of biomedical data. Han et al [137] proposed a data augmentation method called 3D Multi-Conditional GAN (MCGAN), and it can help to overcome medical data paucity.…”

Section: B Future Opportunitiesmentioning

confidence: 99%

A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks

et al. 2020

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have achieved impressive results in various image synthesis tasks, and are becoming a hot topic in computer vision research because of the impressive performance they achieved in various applications. In this paper, we introduce the recent research on GANs in the field of image processing, including image synthesis, image generation, image semantic editing, image-to-image translation, image super-resolution, image inpainting, and cartoon generation. We analyze and summarize the methods used in these applications which have improved the generated results. Then, we discuss the challenges faced by GANs and introduce some methods to deal with these problems. We also preview some likely future research directions in the field of GANs, such as video generation, facial animation synthesis and 3D face reconstruction. The purpose of this review is to provide insights into the research on GANs and to present the various applications based on GANs in different scenarios. INDEX TERMS Generative adversarial networks, image synthesis, image-to-image translation, image editing, cartoon generation.

show abstract

“…However, using the developed GUI, removing undesirable edits is made extremely easy, and new edits (i.e., edits in different locations and/or of different sizes) can be made until the user is satisfied with the look of the resulting tumor. For future work, adjusting the method to make use of conditional informationas was done in other related projects [14][15][16] would allow for the generation of more customizable tumors.…”

Section: D Limitationsmentioning

confidence: 99%

Technical Note: Synthesizing of lung tumors in computed tomography images

O'Briain

Bazalova‐Carter

2020

Medical Physics

View full text Add to dashboard Cite

Purpose: When investigating new radiation therapy techniques in the treatment planning stage, it can be extremely time consuming to locate multiple patient scans that match the desired characteristics for the treatment. With the help of machine learning, we propose to bypass the difficulty in finding patient computed tomography (CT) scans that match the treatment requirements. Furthermore, we aim to provide the developed method as a tool that is easily accessible to interested researchers. Methods: We propose a generative adversarial network (GAN) to edit individual volumes of interest (VOIs) in pre-existing CT scans, translating features of the healthy VOIs into features of cancerous volumes. Training and testing was done using VOIs from a dataset of 460 diagnostic and lung cancer screening CT scans. Agreement between real tumors and those produced by the editor was tested by comparing the distributions of several histogram parameters and second-order statistics as well as using qualitative analysis. Results: After training, the network was successfully able to map healthy CT segments to realistic looking cancerous volumes. Based on visual inspection, tumors produced by the editor were found to be both realistic and visually consistent with the surrounding anatomy when placed back into the original CT scan. Furthermore, the network was found to be able to extrapolate well beyond the upper size limit of the training set. Lastly, a graphical user interface (GUI) was developed to easily interact with the resulting network. Conclusion: The trained network and associated GUI can serve as a tool to develop an abundance of lung cancer patient data to be used in treatment planning. In addition, this method can be extended to a variety of cancer types if given an appropriate baseline dataset.

show abstract

Synthesizing Diverse Lung Nodules Wherever Massively: 3D Multi-Conditional GAN-Based CT Image Augmentation for Object Detection

Cited by 106 publications

References 36 publications

A Hybrid End-to-End Approach Integrating Conditional Random Fields into CNNs for Prostate Cancer Detection on MRI

A Hybrid End-to-End Approach Integrating Conditional Random Fields into CNNs for Prostate Cancer Detection on MRI

A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks

Technical Note: Synthesizing of lung tumors in computed tomography images

Contact Info

Product

Resources

About