Medical Image Synthesis for Data Augmentation and Anonymization Using Generative Adversarial Networks

Shin, Hoo Chang; Tenenholtz, Neil A.; Rogers, Jameson K.; Schwarz, Christopher G.; Senjem, Matthew L.; Gunter, Jeffrey L.; Andriole, Katherine P.; Michalski, Mark

doi:10.1007/978-3-030-00536-8_1

Cited by 421 publications

(310 citation statements)

References 23 publications

Supporting

Mentioning

305

Contrasting

Unclassified

Order By: Relevance

“…These issues are exacerbated by data‐hungry methods, including deep neural networks. Unsupervised and self‐supervised methods do not require explicit labeling and hence promise to alleviate some of these issues, whereas synthetic data can potentially enable a faster route toward curation, address the inevitable class imbalance, and mitigate patient privacy concerns. Standardized benchmarking is of particular importance in the medical domain, especially given the multitude of imaging modalities and anatomic sites, as well as acquisition standards and hardware.…”

Section: Challenges and Future Directionsmentioning

confidence: 99%

Artificial intelligence in cancer imaging: Clinical challenges and applications

Hosny

Schabath

et al. 2019

CA A Cancer J Clinicians

1,051

820

View full text Add to dashboard Cite

Judgement, as one of the core tenets of medicine, relies upon the integration of multilayered data with nuanced decision making. Cancer offers a unique context for medical decisions given not only its variegated forms with evolution of disease but also the need to take into account the individual condition of patients, their ability to receive treatment, and their responses to treatment. Challenges remain in the accurate detection, characterization, and monitoring of cancers despite improved technologies. Radiographic assessment of disease most commonly relies upon visual evaluations, the interpretations of which may be augmented by advanced computational analyses. In particular, artificial intelligence (AI) promises to make great strides in the qualitative interpretation of cancer imaging by expert clinicians, including volumetric delineation of tumors over time, extrapolation of the tumor genotype and biological course from its radiographic phenotype, prediction of clinical outcome, and assessment of the impact of disease and treatment on adjacent organs. AI may automate processes in the initial interpretation of images and shift the clinical workflow of radiographic detection, management decisions on whether or not to administer an intervention, and subsequent observation to a yet to be envisioned paradigm. Here, the authors review the current state of AI as applied to medical imaging of cancer and describe advances in 4 tumor types (lung, brain, breast, and prostate) to illustrate how common clinical problems are being addressed. Although most studies evaluating AI applications in oncology to date have not been vigorously validated for reproducibility and generalizability, the results do highlight increasingly concerted efforts in pushing AI technology to clinical use and to impact future directions in cancer care.

show abstract

Section: Challenges and Future Directionsmentioning

confidence: 99%

Artificial intelligence in cancer imaging: Clinical challenges and applications

Hosny

Schabath

et al. 2019

CA A Cancer J Clinicians

1,051

820

View full text Add to dashboard Cite

show abstract

“…In these cases, GANs that have been trained in-house may serve as a mean to distribute the information contained within the database without actually providing a real snapshot of patient sensitive data: only the weight distribution of the GAN needs to be transferred and a representative artificial dataset of millions of radiographs may be generated in reasonable computational time at a peripheral site. This is in contrast to previous works of Shin et al in which lower resolution artificial images could be produced but always required a recourse to the original patient images as inputs to an image to image translational network [35,36]. We demonstrate the feasibility of using GANs as a tool of effective oversampling when the pathology distribution within a medical dataset is highly imbalanced.…”

Section: Discussionmentioning

confidence: 65%

Breaking Medical Data Sharing Boundaries by Employing Artificial Radiographs

Han

Nebelung

Haarburger

et al. 2019

Preprint

View full text Add to dashboard Cite

Artificial intelligence (AI) has the potential to change medicine fundamentally. Here, expert knowledge provided by AI can enhance diagnosis by comprehensive and user independent integration of multiple image features. Unfortunately, existing algorithms often stay behind expectations, as databases used for training are usually too small, incomplete, and heterogeneous in quality. Additionally, data protection constitutes a serious obstacle to data sharing. We propose to use generative models (GM) to produce high-resolution artificial radiographs, which are free of personal identifying information. Blinded analyses by computer vision and radiology experts proved the high similarity of artificial and real radiographs. The combination of multiple GM improves the performance of computer vision algorithms and the integration of artificial data into patient data repositories can compensate for underrepresented disease entities. Furthermore, the low computational effort of our method complies with existing IT infrastructure in hospitals and thus facilitates its dissemination. We envision that our approach could lead to scalable databases of anonymous medical images enabling standardized radiomic analyses at multiple sites.

show abstract

“…In practice, it is often difficult to collect enough training data, especially for a new imaging modality not well established in clinical practice yet. What's more, data with high-class imbalance or insufficient variability [55] often results in poor classification performance. Thus, our model can synthesize more multi-modal images, which can be regarded as supplementary training data to boost the generalization capability of current deep learning models.…”

Section: F Discussionmentioning

confidence: 99%

Hi-Net: Hybrid-Fusion Network for Multi-Modal MR Image Synthesis

Zhou

Chen

et al. 2020

IEEE Trans. Med. Imaging

226

112

View full text Add to dashboard Cite

Magnetic resonance imaging (MRI) is a widely used neuroimaging technique that can provide images of different contrasts (i.e., modalities). Fusing this multi-modal data has proven particularly effective for boosting model performance in many tasks. However, due to poor data quality and frequent patient dropout, collecting all modalities for every patient remains a challenge. Medical image synthesis has been proposed as an effective solution to this, where any missing modalities are synthesized from the existing ones. In this paper, we propose a novel Hybrid-fusion Network (Hi-Net) for multi-modal MR image synthesis, which learns a mapping from multi-modal source images (i.e., existing modalities) to target images (i.e., missing modalities). In our Hi-Net, a modality-specific network is utilized to learn representations for each individual modality, and a fusion network is employed to learn the common latent representation of multi-modal data. Then, a multi-modal synthesis network is designed to densely combine the latent representation with hierarchical features from each modality, acting as a generator to synthesize the target images. Moreover, a layer-wise multi-modal fusion strategy is presented to effectively exploit the correlations among multiple modalities, in which a Mixed Fusion Block (MFB) is proposed to adaptively weight different fusion strategies (i.e., element-wise summation, product, and maximization). Extensive experiments demonstrate that the proposed model outperforms other state-of-the-art medical image synthesis methods.

show abstract

Medical Image Synthesis for Data Augmentation and Anonymization Using Generative Adversarial Networks

Cited by 421 publications

References 23 publications

Artificial intelligence in cancer imaging: Clinical challenges and applications

Artificial intelligence in cancer imaging: Clinical challenges and applications

Breaking Medical Data Sharing Boundaries by Employing Artificial Radiographs

Hi-Net: Hybrid-Fusion Network for Multi-Modal MR Image Synthesis

Contact Info

Product

Resources

About