Review of Generative Adversarial Networks in mono- and cross-modal biomedical image registration

Abstract

Biomedical image registration refers to aligning corresponding anatomical structures among different images, which is critical to many tasks, such as brain atlas building, tumor growth monitoring, and image fusion-based medical diagnosis. However, high-throughput biomedical image registration remains challenging due to inherent variations in the intensity, texture, and anatomy resulting from different imaging modalities, different sample preparation methods, or different developmental stages of the imaged subject. Recently, Generative Adversarial Networks (GAN) have attracted increasing interest in both mono- and cross-modal biomedical image registrations due to their special ability to eliminate the modal variance and their adversarial training strategy. This paper provides a comprehensive survey of the GAN-based mono- and cross-modal biomedical image registration methods. According to the different implementation strategies, we organize the GAN-based mono- and cross-modal biomedical image registration methods into four categories: modality translation, symmetric learning, adversarial strategies, and joint training. The key concepts, the main contributions, and the advantages and disadvantages of the different strategies are summarized and discussed. Finally, we analyze the statistics of all the cited works from different points of view and reveal future trends for GAN-based biomedical image registration studies.

Two‐stage deep‐learning‐based colonoscopy polyp detection incorporating fisheye and reflection correction

“…GANs are often used for image augmentation and restoration. 37,38 For example, Yang et al developed an inpainting method that uses a context encoder to generate the missing parts of images and multiscale neural patch synthesis to refine the textures and details of the generated parts. 39 The system employed in this study did not detect some colon polyps when the image was taken while the lumen was not well distended.…”

“…The networks are constantly competing to update their parameters, ultimately enabling the generative network to generate an output that cannot be distinguished from authentic images by the discriminator network. GANs are often used for image augmentation and restoration 37,38 . For example, Yang et al .…”