Lightweight image super-resolution based on deep learning: State-of-the-art and future directions

Gendy, Garas; He, Guanghui; Sabor, Nabil

doi:10.1016/j.inffus.2023.01.024

Cited by 28 publications

(11 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TCFDN was able to extract refined multi-level features with better representation ability while remaining lightweight. Other efficient hybrid networks can be found in [23].…”

Section: B Cnn-based Srmentioning

confidence: 99%

Semantic Super-Resolution via Self-Distillation and Adversarial Learning

Park

2024

IEEE Access

View full text Add to dashboard Cite

Semantic super-resolution (SR) is an approach that improves the SR performance by leveraging semantic information about the scene. This study develops a novel semantic SR method that is based on the generative adversarial network (GAN) framework and self-distillation. A discriminator is adversarially trained along with a generator to extract semantic features from images and distinguish semantic differences between images. To train the generator, an additional adversarial loss is computed from the discriminator's outputs of SR images belonging to the same category and minimized via self-distillation. This guides the generator to learn implicit category-specific semantic priors. We conducted experiments for SR of text and face images using the Enhanced Deep Super-Resolution (EDSR) generator and the SRGAN discriminator. Experimental results showed that our method can contribute to improving both the quantitative and qualitative quality of SR images. Although the improvement varied depending on image category and dataset, the peak signal-to-noise ratio (PSNR) value increased by up to 0.87 dB and the perceptual index (PI) decreased by up to 0.17 by using our method. Our method outperformed existing semantic SR methods.

show abstract

“…TCFDN was able to extract refined multi-level features with better representation ability while remaining lightweight. Other efficient hybrid networks can be found in [23].…”

Section: B Cnn-based Srmentioning

confidence: 99%

Semantic Super-Resolution via Self-Distillation and Adversarial Learning

Park

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Diferent from the above reconstruction performanceoriented networks, lightweight SISR models focus more on model complexity including model size (i.e., the number of network parameters) and operation numbers (i.e., multiadds) [43]. Tis kind of SISR methods has received increasing attention for its potential application in resourcelimited systems.…”

Section: Lightweight Cnn For Complexity-oriented Sisrmentioning

confidence: 99%

“…Benefting from various network design based on feature distillation [37][38][39] and attention mechanism [40][41][42], recent research on lightweight SISR has made a lot of progress. Nevertheless, how to fully utilize deep features to make a balance between the capacity and complexity of an SR network remains an open problem [43].…”

Section: Introductionmentioning

confidence: 99%

A Multi-Attention Feature Distillation Neural Network for Lightweight Single Image Super-Resolution

Zhang,

Lin,

Yang

et al. 2024

International Journal of Intelligent Systems

View full text Add to dashboard Cite

In recent years, remarkable performance improvements have been produced by deep convolutional neural networks (CNN) for single image super-resolution (SISR). Nevertheless, a high proportion of CNN-based SISR models are with quite a few network parameters and high computational complexity for deep or wide architectures. How to more fully utilize deep features to make a balance between model complexity and reconstruction performance is one of the main challenges in this field. To address this problem, on the basis of the well-known information multi-distillation model, a multi-attention feature distillation network termed as MAFDN is developed for lightweight and accurate SISR. Specifically, an effective multi-attention feature distillation block (MAFDB) is designed and used as the basic feature extraction unit in MAFDN. With the help of multi-attention layers including pixel attention, spatial attention, and channel attention, MAFDB uses multiple information distillation branches to learn more discriminative and representative features. Furthermore, MAFDB introduces the depthwise over-parameterized convolutional layer (DO-Conv)-based residual block (OPCRB) to enhance its ability without incurring any parameter and computation increase in the inference stage. The results on commonly used datasets demonstrate that our MAFDN outperforms existing representative lightweight SISR models when taking both reconstruction performance and model complexity into consideration. For example, for ×4 SR on Set5, MAFDN (597K/33.79G) obtains 0.21 dB/0.0037 and 0.10 dB/0.0015 PSNR/SSIM gains over the attention-based SR model AFAN (692K/50.90G) and the feature distillation-based SR model DDistill-SR (675K/32.83G), respectively.

show abstract

“…To meet the demands of edge devices, it is imperative to develop lightweight and efficient SR models (Gendy, He, and Sabor 2023). Many researchers have designed various lightweight SR algorithms to reduce parameter count and computational complexity.…”

Section: Sisr For Computational Efficiencymentioning

confidence: 99%

OSFFNet: Omni-Stage Feature Fusion Network for Lightweight Image Super-Resolution

Wang,

Zhang

2024

AAAI

View full text Add to dashboard Cite

Recently, several lightweight methods have been proposed to implement single-image super-resolution (SISR) on resource-constrained devices. However, these methods primarily focus on simplifying network structures without the full utilization of shallow features. The fact remains that shallow features encompass crucial details for the super-resolution task, including edges, textures, and colors. Therefore, developing a novel architecture that can effectively integrate features from different levels and capitalize on their mutual complementarity is necessary. We first analyze the relationship between multi-stage features and the restoration tasks in a classic lightweight SR method. Based on these observations, we propose an Omni-Stage Feature Fusion (OSFF) architecture, which incorporates Original Image Stacked Initialisation, Shallow Feature Global Connection, and Multi-Receptive Field Dynamic Fusion. An Attention-Enhanced Feature Distillation module is also designed to enhance the model performance. Finally, leveraging these contributions, we construct an Omni-Stage Feature Fusion Network (OSFFNet). Through extensive experiments on various benchmark datasets, the proposed model outperforms state-of-the-art methods. Notably, it achieves a 0.26dB PSNR improvement over the second-best method for x2 SR on the Urban100 dataset.

show abstract

Lightweight image super-resolution based on deep learning: State-of-the-art and future directions

Cited by 28 publications

References 40 publications

Semantic Super-Resolution via Self-Distillation and Adversarial Learning

Semantic Super-Resolution via Self-Distillation and Adversarial Learning

A Multi-Attention Feature Distillation Neural Network for Lightweight Single Image Super-Resolution

OSFFNet: Omni-Stage Feature Fusion Network for Lightweight Image Super-Resolution

Contact Info

Product

Resources

About