Multi-scale frequency separation network for image deblurring

Abstract

Image deblurring aims to restore the detailed texture information or structures from the blurry images, which has become an indispensable step in many computer-vision tasks. Although various methods have been proposed to deal with the image deblurring problem, most of them treated the blurry image as a whole and neglected the characteristics of different image frequencies. In this paper, we present a new method called multi-scale frequency separation network (MSFS-Net) for image deblurring. MSFS-Net introduces the frequency separation module (FSM) into an encoder-decoder network architecture to capture the lowand high-frequency information of image at multiple scales. Then, a simple cycle-consistency strategy and a sophisticated contrastive learning module (CLM) are respectively designed to retain the low-frequency information and recover the high-frequency information during deblurring. At last, the features of different scales are fused by a crossscale feature fusion module (CSFFM). Extensive experiments on benchmark datasets show that the proposed network achieves state-of-the-art performance.

Dual-Domain Attention for Image Deblurring

“…On the other hand, most of the recent deep learning-based methods solve deblurring only in the spatial domain without sufficiently utilizing the discrepancies in the frequency domain. Recently, some works have been proposed to reduce the frequency gap between sharp/blurry image pairs (Zou et al 2021;Mao et al 2021;Liu et al 2020;Zhang et al 2022b). For instance, SDWNet (Zou et al 2021) introduces wavelet reconstruction module to decouple the features into various frequency subbands by Wavelet transform, which needs additional computational complexity to perform inverse transform.…”

“…However, these two branches share the same input, and as a consequence, the low-frequency information conveyed to residual branch may disturb the learning of high-frequency signal. MSFS-Net (Zhang et al 2022b) adopts multiple complicated OctConv (Chen et al 2019) to conduct frequency separation, and down/upsampling operations are performed frequently, introducing extra computation burdens.…”

“…Deep-RFT leverages FFT-based branch to process frequency information (Mao et al 2021). MSFS-Net adopts multiple Oct-Conv (Chen et al 2019) operation to decompose frequency (Zhang et al 2022b). We explore a simpler frequency attention module to first decouple the spectrum with average pooling technique, and then emphasize the useful frequency components and suppress the less informative ones via learnable parameters.…”