How Can We Make Gan Perform Better in Single Medical Image Super-Resolution? A Lesion Focused Multi-Scale Approach

Abstract

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

A new deep convolutional neural network design with efficient learning capability: Application to CT image synthesis from MRI

“…In addition to the above mentioned methods, generative adversarial networks (GAN) have shown great potential in a broad range of applications, including image reconstruction, 31,32 partial volume correction, 33 and superresolution imaging. 34 In this regard, the adversarial semantic structure deep learning proposed in Ref. [35] resulted in reliable synthetic CT generation and clinically tolerable PET quantification bias.…”

Efficient Reconstruction Technique for Multi-Slice CS-MRI Using Novel Interpolation and 2D Sampling Scheme

“…The classic CS-MRI uses fixed sparsifying transforms like total variation (TV) [32], discrete cosine transforms (DCT) and discrete wavelet transforms (DWT) [33]. On the other hand, with the rapid development of deep learning, many CNN-based super-resolution (SR) methods [34][35][36] have also gained attention in recent years. In CS-MRI all slices should be equally under-sampled and are then recovered using some non-linear reconstruction algorithms [2,13,37].…”

Consecutive Context Perceive Generative Adversarial Networks for Serial Sections Inpainting

“…Dai et al [33] and Jeon et al [34] propose to learn spatially attentive or active convolutional kernels, but may still be limited in exacting features from the background. Recently, Zhu et al [35] introduced lesion focused SR (LFSR) and multi-scale method to improve perceptual quality of the super-resolved results for brain tumor MRI images.…”