Efficient Attention Fusion Network in Wavelet Domain for Demoireing

Abstract

When taking pictures of electronic screens or objects with high-frequency textures, people often run across colorful rainbow patterns that are known as ''moire'', seriously affecting the image quality and subsequent processing. Current methods for removing moire patterns mostly extract multiscale information by downsampling pooling layers, which may inevitably cause information loss. To address this issue, this paper proposes a demoireing method in the wavelet domain. By employing both discrete wavelet transform (DWT) and inverse discrete wavelet transform (IDWT) instead of traditional downsampling and upsampling, this method can effectively increase the network receptive field without information loss. In addition, to further reconstruct more details of moire patterns, this paper proposes an efficient attention fusion module (EAFM). With a combination of efficient channel attention, spatial attention and local residual learning, this module can self-adaptively learn various weights of feature information at different levels and inspire the network to focus more on effective information such as moire details to improve learning and demoireing performance. Extensive experiments based on public datasets have shown that this suggested method can efficiently remove moire patterns and has a good quantitative and qualitative performance.INDEX TERMS Demoire, deep learning, wavelet transform, attention mechanism.

Unpaired Screen-Shot Image Demoiréing With Cyclic Moiré Learning

“…In addition, attempts have been made to remove moiré artifacts in the frequency domain. For example, in [32], [33], demoiréing was performed in the wavelet transform domain. Zheng et al [1] addressed the diversity of moiré artifacts by developing learnable bandpass filters.…”

Deep Learning for Screen-Shot Image Demoiréing: A Survey

“…Because residual learning only learns the variation in the difference between clean and moiré images without learning the complete transformation between images, it reduces learning difficulty and complexity [22]. In addition, residual learning can avoid the gradient diffusion problem caused by adding layers to the network and increasing the network's performance and stability [15]. In practical applications, as well as stacking residual blocks, residual learning is used in combination with the attention mechanism [14], [17], [42] and dense connectivity [45], [46] to address the dynamic properties of moiré pattern and the variability between different branches.…”

“…Guo et al [17] combined channel attention and spatial attention and used channel-level attention maps to guide the production of spatial attention maps that accurately find the most critical information in multi-scale features. Sun et al [15] proposed an efficient attention fusion module that combines channel attention, spatial attention, and local residual learning, adaptively learning the different features of…”

“…where H , W , and C are the image length, width, and the number of image channels, respectively. Besides, a variant of frequently used L1 loss, L1 Charbonnier loss [15], [16], [18], [42], [46]:…”

“…Researchers have proposed a variety of deep learning-based demoiréing models, ranging from convolutional neural networks (CNN) (e.g., DMCNN [10]) to generative adversarial nets (GAN) [11](e.g., MR-GAN [12] and cyclic GAN [13]), can remove moiré within 1s, and achieved the best score in image quality evaluation. In general, the methods of image demoiréing with deep learning differ in the following aspects: sampling methods [10], [14], [15], network design [14], [16], [17], baseline model [15], [16], [18], [21], [60], [61], and learning strategies [12], [18]- [22].…”