DDistill-SR: Reparameterized Dynamic Distillation Network for Lightweight Image Super-Resolution

Abstract

Recent research on deep convolutional neural networks (CNNs) has provided a significant performance boost on efficient super-resolution (SR) tasks by trading off the performance and applicability. However, most existing methods focus on subtracting feature processing consumption to reduce the parameters and calculations without refining the immediate features, which leads to inadequate information in the restoration. In this paper, we propose a lightweight network termed DDistill-SR, which significantly improves the SR quality by capturing and reusing more helpful information in a static-dynamic feature distillation manner. Specifically, we propose a plug-in reparameterized dynamic unit (RDU) to promote the performance and inference cost trade-off. During the training phase, the RDU learns to linearly combine multiple reparameterizable blocks by analyzing varied input statistics to enhance layerlevel representation. In the inference phase, the RDU is equally converted to simple dynamic convolutions that explicitly capture robust dynamic and static feature maps. Then, the information distillation block is constructed by several RDUs to enforce hierarchical refinement and selective fusion of spatial context information. Furthermore, we propose a dynamic distillation fusion (DDF) module to enable dynamic signals aggregation and communication between hierarchical modules to further improve performance. Empirical results show that our DDistill-SR outperforms the baselines and achieves state-of-the-art results on most super-resolution domains with much fewer parameters and less computational overhead. We have released the code of DDistill-SR at https://github.com/icandle/DDistill-SR.

F2SRGAN: A Lightweight Approach Boosting Perceptual Quality in Single Image Super-Resolution via a Revised Fast Fourier Convolution

“…Table 1 summarizes the quantitative comparison between the proposed F2SRGAN with current state-of-the-art lightweight SISR methods, including DDistill-SR [29], FDSCSR-S [30], FRN [31], HNCT [32], HPINet-M [33], LatticeNet [34], LESRCNN [35], OverNet [36], SCET [37], SRRFN-BI [38], URN [39], and SwiftSRGAN [1]. We report PI and PSNR, alongside performance metrics, which are total numbers of parameters and multiply-adds counter on the upscale factor of 2 and 4.…”

Towards Real-Time 4K Image Super-Resolution

“…As shown in Figure 1, our network significantly outperforms the state-of-the-art (SOTA) efficient models such as DDistill-SR (Wang et al, 2022) or SAFMN (Sun et al, 2023) by a considerable margin, while utilizing only half or even less of the GMACS. Although our model is specifically designed for efficient SR, its scalability is evident as our larger model surpasses the SOTA lightweight transformer in performance while incurring lower computational costs.…”

“…We present quantitative results for ×2, ×3, and ×4 image SR, comparing against current efficient state-of-the-art models in Table 1, including CARN-M (Ahn et al, 2018), IMDN (Hui et al, 2019), PAN (Zhao et al, 2020), DR-SAN (Park et al, 2021), DDistill-SR (Wang et al, 2022), ShuffleMixer (Sun et al, 2022), and SAFMN (Sun et al, 2023). Additionally, we evaluate against lightweight variants of popular Transformer-based SR models such as SwinIR (Liu et al, 2021), ELAN (Zhang et al, 2022), and SRFormer (Zhou et al, 2023) may suffer blurring artifacts, distortions, or inaccurate texture restoration.…”

“…In recent years, the pursuit of efficient SR techniques has gained significant momentum (Li et al, 2022;Ignatov et al, 2023;Li et al, 2023;Conde et al, 2023). Consequently, researchers have introduced streamlined neural architectures (Ignatov et al, 2021), network compression (Wang et al, 2022), reparameterization (Zhang et al, 2021b), and other training strategies to cater to the demand for efficiency. Initially, efficient SR methods utilized group convolutions and cascaded block designs to boost efficiency (Ahn et al, 2018;Hui et al, 2019).…”

Depthwise channel attention network (DWCAN): An efficient and lightweight model for single image super‐resolution and metaverse gaming

“…An SRB is additionally introduced, the core component of RFDN, allowing the network to efficiently leverage the residual learning while maintaining its lightweight nature. Inspired by RFDN, another approach has been recently proposed, known as a reparametrized dynamic distillation Network (Wang et al, 2022), which introduces reparametrized dynamic unit (RDU) and dynamic distillation fusion (DDF) to raise the performance and to qualify dynamic signals collection to tradeoff the running cost, respectively. The importance of SR can also be seen in segmentation for example, an adaptive multi‐scale dual attention network (Wang, Wang, et al, 2021) is proposed for semantic segmentation.…”