A Novel Un-Supervised GAN for Fundus Image Enhancement with Classification Prior Loss

Abstract

Fundus images captured for clinical diagnosis usually suffer from degradation factors due to variation in equipment, operators, or environment. These degraded fundus images need to be enhanced to achieve better diagnosis and improve the results of downstream tasks. As there is no paired low- and high-quality fundus image, existing methods mainly focus on supervised or semi-supervised learning methods for color fundus image enhancement (CFIE) tasks by utilizing synthetic image pairs. Consequently, domain gaps between real images and synthetic images arise. With respect to existing unsupervised methods, the most important low scale pathological features and structural information in degraded fundus images are prone to be erased after enhancement. To solve these problems, an unsupervised GAN is proposed for CFIE tasks utilizing adversarial training to enhance low quality fundus images. Synthetic image pairs are no longer required during the training. A specially designed U-Net with skip connection in our enhancement network can effectively remove degradation factors while preserving pathological features and structural information. Global and local discriminators adopted in the GAN lead to better illumination uniformity in the enhanced fundus image. To better improve the visual quality of enhanced fundus images, a novel non-reference loss function based on a pretrained fundus image quality classification network was designed to guide the enhancement network to produce high quality images. Experiments demonstrated that our method could effectively remove degradation factors in low-quality fundus images and produce a competitive result compared with previous methods in both quantitative and qualitative metrics.

Computational single fundus image restoration techniques: a review

“…( 96 ) proposed a new mathematical model to formulate the image-degrading process of fundus imaging and train a network for retinal image restoration. Others have modified the structures of the network or loss function to improve the performance of the networks ( 97 , 98 ).…”

YOLO Based Deep Learning Model for Segmenting the Color Images

“…For instance, figure 5 shows the three rows, where input image in first row, ground truth image in second row and finally, segmented images using proposed model in third row. [22,27], RNN [39], CNN [41], GAN [42], YOLO [26,27], CSO [31] and SSA [31] are all tested with these two datasets and results are mentioned in the following tables.…”

RFormer: Transformer-Based Generative Adversarial Network for Real Fundus Image Restoration on a New Clinical Benchmark

“…Generative Adversarial Network (GAN) is firstly introduced in [45] and has been proven successful in image synthesis [16]- [18], and translation [17], [18]. Subsequently, GAN is applied to image restoration and enhancement, e.g., super resolution [19]- [25], deraining [26], deblurring [27], enlighten [32], [33], dehazing [46], [47], image inpainting [48], [49], style transfer [18], [50], image editing [51], [52], medical image enhancement [22], [23], [53], [54], and mobile photo enhancement [55], [56]. Although GAN is widely applied in low-level vision tasks, few works are dedicated to improving the underlying framework of GAN, such as replacing the traditional CNN framework with Transformer.…”