AdderSR: Towards Energy Efficient Image Super-Resolution

Song, Dehua; Wang, Yunhe; Chen, Hanting; Xu, Chang; Xu, Chunjing; Tao, Dacheng

doi:10.1109/cvpr46437.2021.01539

Cited by 72 publications

(24 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image super-resolution is an important image restoration task in computer vision [8,21,23,[29][30][31]. Since SRCNN [5] first applied early convolutional neural networks to solve SR tasks.…”

Section: Image Super-resolutionmentioning

confidence: 99%

What Hinders Perceptual Quality of PSNR-oriented Methods?

Xu¹,

Mi²,

Zheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

PSNR-oriented models (POMs) have achieved great success and are widely adopted in many image superresolution (SR) applications. However, details of images generated by these models are usually over-smoothed. Many existing methods improve the perceptual quality of generated images by designing models with larger sizes to optimize the models' learning ability, or using GAN-based models to generate fake details. Unfortunately, those methods cost too much labor effort to be practical in real-world applications. To solve this issue, in this paper, we discover two factors that inhibit POMs from achieving high perceptual quality: 1) center-oriented optimization (COO) problem and 2) model's low-frequency tendency. First, POMs tend to generate an SR image whose position in the feature space is closest to the distribution center of all potential high-resolution (HR) images, resulting in such POMs losing high-frequency details. Second, 90% area of an image consists of low-frequency signals; in contrast, human perception relies on an image's high-frequency details. However, POMs apply the same calculation to process differentfrequency areas, so that POMs tend to restore the lowfrequency regions. Based on these two factors, we propose a Detail Enhanced Contrastive Loss (DECLoss), by combining a high-frequency enhancement module and spatial contrastive learning module, to reduce the influence of the COO problem and low-Frequency tendency. Experimental results show the efficiency and effectiveness when applying DECLoss on several regular SR models. E.g, in EDSR, our proposed method achieves 3.60× faster learning speed compared to a GAN-based method with a subtle degradation in visual quality. In addition, our final results show that an SR network equipped with our DECLoss generates more realistic and visually pleasing textures compared to state-of-the-art methods.

show abstract

Section: Image Super-resolutionmentioning

confidence: 99%

What Hinders Perceptual Quality of PSNR-oriented Methods?

Xu¹,

Mi²,

Zheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Among various research areas of AI, CV is a longstanding and fundamental field, which allows computers to derive meaningful information from digital images, videos, and other visual inputs. As a representative method in CV, CNN models have achieved the new SOTA performance on a wide range of tasks over the last few decades, e.g., Image Recognition [36], Object Detection [37]- [39], Image Segmentation [40]- [42], and Image Processing [43]- [46]. Image recognition involves analyzing images and identifying objects, actions, and other elements in order to draw conclusions.…”

Section: Computer Visionmentioning

confidence: 99%

“…Great success has been achieved in many other CV tasks [68], such as super-resolution [43], image restoration [69], [70], and image generation [71]- [73]. For example, DALL-E [71] trains a 12-billion parameter autoregressive transformer for zero-shot text-to-image generation, and observe improved generalization on out-of-domain datasets.…”

Section: Computer Visionmentioning

confidence: 99%

Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI

Yao¹,

Zhang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Influenced by the great success of deep learning via cloud computing and the rapid development of edge chips, research in artificial intelligence (AI) has shifted to both of the computing paradigms, i.e., cloud computing and edge computing. In recent years, we have witnessed significant progress in developing more advanced AI models on cloud servers that surpass traditional deep learning models owing to model innovations (e.g., Transformers, Pretrained families), explosion of training data and soaring computing capabilities. However, edge computing, especially edge and cloud collaborative computing, are still in its infancy to announce their success due to the resource-constrained IoT scenarios with very limited algorithms deployed. In this survey, we conduct a systematic review for both cloud and edge AI. Specifically, we are the first to set up the collaborative learning mechanism for cloud and edge modeling with a thorough review of the architectures that enable such mechanism. We also discuss potentials and practical experiences of some on-going advanced edge AI topics including pretraining models, graph neural networks and reinforcement learning. Finally, we discuss the promising directions and challenges in this field.

show abstract

“…Single image super-resolution (SISR) is a classical computer vision problem that tries to infer a high-resolution (HR) image from a single low-resolution (LR) input image. This problem is still an active research field in the computer vision community (e.g., [ 1 , 2 , 3 , 4 ]). Several applications in different fields can benefit from super-resolution (SR) representations, for instance, security (e.g., [ 5 , 6 ]), medical imaging (e.g., [ 7 ]), object detection (e.g., [ 8 ]), and astronomical images (e.g., [ 9 ]), among others.…”

Section: Introductionmentioning

confidence: 99%

A Novel Domain Transfer-Based Approach for Unsupervised Thermal Image Super-Resolution

Rivadeneira

Sappa

Vintimilla

et al. 2022

Sensors

View full text Add to dashboard Cite

This paper presents a transfer domain strategy to tackle the limitations of low-resolution thermal sensors and generate higher-resolution images of reasonable quality. The proposed technique employs a CycleGAN architecture and uses a ResNet as an encoder in the generator along with an attention module and a novel loss function. The network is trained on a multi-resolution thermal image dataset acquired with three different thermal sensors. Results report better performance benchmarking results on the 2nd CVPR-PBVS-2021 thermal image super-resolution challenge than state-of-the-art methods. The code of this work is available online.

show abstract

AdderSR: Towards Energy Efficient Image Super-Resolution

Cited by 72 publications

References 22 publications

What Hinders Perceptual Quality of PSNR-oriented Methods?

What Hinders Perceptual Quality of PSNR-oriented Methods?

Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI

A Novel Domain Transfer-Based Approach for Unsupervised Thermal Image Super-Resolution

Contact Info

Product

Resources

About