Revisiting Image Deblurring with an Efficient ConvNet

Abstract

Image deblurring aims to recover the latent sharp image from its blurry counterpart and has a wide range of applications in computer vision. The Convolution Neural Networks (CNNs) have performed well in this domain for many years, and until recently an alternative network architecture, namely Transformer, has demonstrated even stronger performance. One can attribute its superiority to the multi-head self-attention (MHSA) mechanism, which offers a larger receptive field and better input content adaptability than CNNs. However, as MHSA demands high computational costs that grow quadratically with respect to the input resolution, it becomes impractical for high-resolution image deblurring tasks. In this work, we propose a unified lightweight CNN network that features a large effective receptive field (ERF) and demonstrates comparable or even better performance than Transformers while bearing less computational costs. Our key design is an efficient CNN block dubbed LaKD, equipped with a large kernel depth-wise convolution and spatial-channel mixing structure, attaining comparable or larger ERF than Transformers but with a smaller parameter scale. Specifically, we achieve +0.17dB / +0.43dB PSNR over the state-of-theart Restormer on defocus / motion deblurring benchmark datasets with 32% fewer parameters and 39% fewer MACs. Extensive experiments demonstrate the superior performance of our network and the effectiveness of each module. Furthermore, we propose a compact and intuitive ERFMeter metric that quantitatively characterizes ERF, and shows a high correlation to the network performance. We hope this work can inspire the research community to further explore the pros and cons of CNN and Transformer architectures beyond image deblurring tasks.

Omni-Kernel Network for Image Restoration

“…We verify the effectiveness of the proposed network for single-image defocus deblurring using the widely used DPDD (Abuolaim and Brown 2020) dataset, and compare the results with 7 representative algorithms: DPDNet (Abuolaim and Brown 2020), KPAC (Son et al 2021), DRBNet (Ruan et al 2022), IFAN (Lee et al 2021), MDP (Abuolaim, Afifi, and Brown 2022), Restormer (Zamir et al 2022), and LaKDNet (Ruan et al 2023). The results are shown in Table 3.…”

“…SLaK (Liu et al 2023) leverages sparse factorized 51 × 51 kernels to confront Transformer methods. In the realm of image restoration, LaKDNet (Ruan et al 2023) enlarges the effective receptive field via combinations of large kernel (9×9) depth-wise convolutions and point-wise convolutions. MAN (Wang et al 2022b) presents the large kernel attention by decomposing a large kernel convolution into three different kinds of convolutions.…”

“…In the context of image restoration, LKDNet (Luo et al 2022) decomposes a 21×21 convolution into a smaller depth-wise convolution and a depth-wise dilated convolution for image dehazing. LaKDNet (Ruan et al 2023) leverages large kernel size convolutions (e.g., 9 × 9) followed by point-wise convolutions to obtain large effective receptive fields for image deblurring. MAN (Wang et al 2022b) decomposes a large kernel size convolution into three components, i.e., a depth-wise convolution, a depth-wise dilated…”