Learning to Clean: A GAN Perspective

Abstract

In the big data era, the impetus to digitize the vast reservoirs of data trapped in unstructured scanned documents such as invoices, bank documents, courier receipts and contracts has gained fresh momentum. The scanning process often results in the introduction of artifacts such as salt-and-pepper / background noise, blur due to camera motion or shake, watermarkings, coffee stains, wrinkles, or faded text. These artifacts pose many readability challenges to current text recognition algorithms and significantly degrade their performance. Existing learning based denoising techniques require a dataset comprising of noisy documents paired with cleaned versions of the same document. In such scenarios, a model can be trained to generate clean documents from noisy versions. However, very often in the real world such a paired dataset is not available, and all we have for training our denoising model are unpaired sets of noisy and clean images. This paper explores the use of Generative Adversarial Networks (GAN) to generate denoised versions of the noisy documents. In particular, where paired information is available, we formulate the problem as an image-to-image translation task i.e, translating a document from noisy domain ( i.e., background noise, blurred, faded, watermarked ) to a target clean document using Generative Adversarial Networks (GAN). However, in the absence of paired images for training, we employed CycleGAN which is known to learn a mapping between the distributions of the noisy images to the denoised images using unpaired data to achieve image-to-image translation for cleaning the noisy documents. We compare the performance of CycleGAN for document cleaning tasks using unpaired images with a Conditional GAN trained on paired data from the same dataset. Experiments were performed on a public document dataset on which different types of noise were artificially induced, results demonstrate that CycleGAN learns a more robust mapping from the space of noisy to clean documents.

Unsupervised Feature Elimination via Generative Adversarial Networks: Application to Hair Removal in Melanoma Classification

“…The proposed algorithm is evaluated on the real-world dermoscopic dataset, The ISIC 2020 Challenge Dataset [76]. Due to the lack of prior data-driven unsupervised techniques that aim to eliminate hair from dermoscopic images, Cycle- GAN [42] which has been actively used for unsupervised feature elimination for denosing [39], dehazing [45], [46] and deraining [44], [77] as discussed in Sec. II and UNet+L2 [43] which is the most similar approach to ours are used as benchmarks.…”

“…More recently, deep learning has been employed for feature elimination tasks and fine-grained annotations are used as ground-truth for supervised learning [16], [23], [23]- [40]. Some supervised approaches combine GAN for reducing artifacts [36], [40] and augmenting paired training data with undesirable features and ground-truth clean images [37]- [39]. The development of CycleGAN [42], which is originally proposed for unpaired translation between two different image domains based on cycle-consistency loss, has inspired unsupervised techniques in feature elimination tasks [45], [46].…”

“…With the advent of a generative adversarial network (GAN) [41] and CycleGAN [42], data-driven unsupervised techniques where undesirable features are removed by imposing the distribution of clean images to the reconstructed output via adversarial loss have been developed [39], [43]- [46]. Hair is an undesirable factor that makes visual degradations and causes serious occlusions in dermoscopic images.…”

Using Generative Adversarial Networks for Extraction of Insar Signals From Large-Scale Sentinel-1 Interferograms by Improving Tropospheric Noise Correction

“…The transitivity property of cyclicconsistency loss allows CycleGAN to perform well on unpaired translation. CycleGANs are also known to perform very well for background noise reduction in blurry images (Sharma et al, 2019) and so, it may be of interest to see whether the Cycl-eGAN can perform better than the Conditional GAN for noise removal in inteferograms.…”

End-to-End Unsupervised Document Image Blind Denoising

“…However, all these methods, including DE-GAN need noisy/clean pairs generated by adding the corruption to the clean pages/patches and train one model per artifact type. On the other hand, Sharma et al [21] proposed document image cleansing based on cycle-consistent GANs [32]. This approach theoretically does not require noisy/clean pairs.…”