MSSNet: Multi-Scale-Stage Network for Single Image Deblurring

Abstract

Most of traditional single image deblurring methods before deep learning adopt a coarse-to-fine scheme that estimates a sharp image at a coarse scale and progressively refines it at finer scales. While this scheme has also been adopted to several deep learning-based approaches, recently a number of single-scale approaches have been introduced showing superior performance to previous coarseto-fine approaches both in quality and computation time, making the traditional coarse-to-fine scheme seemingly obsolete. In this paper, we revisit the coarse-to-fine scheme, and analyze defects of previous coarse-to-fine approaches that degrade their performance. Based on the analysis, we propose Multi-Scale-Stage Network (MSSNet), a novel deep learning-based approach to single image deblurring that adopts our remedies to the defects. Specifically, MSS-Net adopts three novel technical components: stage configuration reflecting blur scales, an inter-scale information propagation scheme, and a pixel-shuffle-based multi-scale scheme. Our experiments show that MSSNet achieves the state-of-the-art performance in terms of quality, network size, and computation time.

Revisiting Image Deblurring with an Efficient ConvNet

“…Table 4 shows that we achieve a competitive performance on the GoPro and HIDE datasets, significantly surpassing the existing CNN, GAN, and RNN solutions. For instance, we outperform the latest MSSNet [28] and NAFNet [9] (CNN) by +0.34dB / +0.27dB on the GoPro in terms of PSNR. Our method also outperforms Transformerbased Uformer / Restormer by +0.30dB / +0.43dB on the GoPro dataset, while sparing up to 66.4% / 34.6% parameters and 47.5% / 43.3% MACs.…”

“…However, these two-step strategies are less effective in terms of quality and computational cost due to the error propagation occurring in their iterative optimization procedure. Later, CNN-based methods predominate in the image deblurring task by offering end-to-end solutions [2,11,28,32,43,48,53,63,85,86], and efficiently achieving remarkable results, most of which are tailored to a specific type of blur -motion or defocus. Transformer-based structures (e.g.…”

“…(1) removing the inner shortcut inside the feature mixing module (labeled in red in Fig. 2); (2) removing the middle shortcut starting from the beginning of the LaKD block till [88], MTRNN [45], DMPHN [86], MIMO [11], MPRNet [85], MSSNet [28], NAFNet [9], Uformer [75], and Restormer [84]. All the methods are evaluated using the code provided by their respective authors without any test-time refinement (e.g.…”

“…In the last two decades, CNNs have become a promising tool for image deblurring tasks. Given a large dataset, CNNs have the ability to learn the corresponding priors, which then can be used for image deblurring at inference [2,11,28,32,43,53,63,85,86], showing high efficiency and generalization ability [57]. While on the one hand, the inherent inductive biases contribute to the efficiency of the network; on the other hand, it limits the network's ability to model long-range spatial dependencies.…”

“…Figure 7. The visual comparison between our method and the recent motion deblurring techniques including DBGAN[88], MTRNN[45], DMPHN[86], MIMO[11], MPRNet[85], MSSNet[28], NAFNet[9], Uformer[75], and Restormer[84]. All the methods are evaluated using the code provided by their respective authors without any test-time refinement (e.g.…”