Unpaired Screen-Shot Image Demoiréing With Cyclic Moiré Learning

Abstract

We propose an end-to-end unpaired learning approach to screen-shot image demoiréing based on cyclic moiré learning. The proposed cyclic moiré learning algorithm consists of the moiréing network and the demoiréing network. The moiréing network generates moiré images to construct a pseudo-paired set of moiré and clean images. Then, the demoiréing network is trained in a supervised manner using the generated pseudo-paired dataset to remove moiré artifacts. In the moiréing network, the moiré generation is separately learned as global pixel intensity degradation and moiré pattern generation for more realistic moiré artifact generation. Furthermore, the moiréing network and the demoiréing network are integrated together to be trained in an end-to-end manner. Experimental results on different datasets demonstrate that the proposed algorithm significantly outperforms state-of-the-art unsupervised demoiréing algorithms as well as image restoration algorithms.

Deep Learning for Screen-Shot Image Demoiréing: A Survey

“…Inspired by Cycle GAN [66] and conditional generative adversarial network (CGAN) [67], Park et al [64] proposed a cyclic GAN model that includes a moiré generation network and demoiréing network. G I .…”

“…G I . To further improve the demoiréing capability of cyclic GAN, Park et al [13] improved on [64]. Park et al [13] found that the appearance of moiré in screen-shot images is accompanied by global pixel intensity degradation.…”

“…To further improve the demoiréing capability of cyclic GAN, Park et al [13] improved on [64]. Park et al [13] found that the appearance of moiré in screen-shot images is accompanied by global pixel intensity degradation. When using the generator to generate the moiré pattern, a 256dimensional image histogram was used to estimate the global pixel intensity degradation degree and added to the moiré generator network to make the generated moiré more realistic.…”

“…For example, Fang et al [9] take 10s to process an image with 256  256 resolution, so it requires faster methods. Researchers have proposed a variety of deep learning-based demoiréing models, ranging from convolutional neural networks (CNN) (e.g., DMCNN [10]) to generative adversarial nets (GAN) [11](e.g., MR-GAN [12] and cyclic GAN [13]), can remove moiré within 1s, and achieved the best score in image quality evaluation. In general, the methods of image demoiréing with deep learning differ in the following aspects: sampling methods [10], [14], [15], network design [14], [16], [17], baseline model [15], [16], [18], [21], [60], [61], and learning strategies [12], [18]- [22].…”

“…Differential shooting principle. With the rapid development of deep learning techniques in the past few years, deep learning-based image demoiréing methods have been actively explored.Researchers have proposed a variety of deep learning-based demoiréing models, ranging from convolutional neural networks (CNN) (e.g., DMCNN[10]) to generative adversarial nets (GAN)[11](e.g., MR-GAN[12] and cyclic GAN[13]), can remove moiré within 1s, and achieved the best score in image quality evaluation. In general, the methods of image demoiréing with deep learning differ in the following aspects: sampling methods[10],[14],[15], network design[14],[16],[17], baseline model[15],[16],[18],[21],[60],[61], and learning strategies[12],[18]-[22].While there is a screen-shots image demoiréing survey[23], only two methods are involved, which is not comprehensive.…”