AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results

Zhang, Kai; Danelljan, Martin; Li, Yawei; Timofte, Radu; Liu, Jie; Tang, Jie; Wu, Guorong; Zhu, Yu; He, Xiaoliang; Xu, Wenjie; Li, Chenghua; Leng, Cong; Cheng, Jian; Wu, Guangyang; Wang, Wenyi; Liu, Xiaohong; Zhao, Hengyuan; Kong, Xiangtao; He, Jingwen; Qiao, Yu; Dong, Chao; Luo, Xiaotong; Chen, Liang; Zhang, Jiangtao; Suin, Maitreya; Purohit, Kuldeep; Rajagopalan, A. N.; Li, Xiaochuan; Lang, Zhiqiang; Nie, Jiangtao; Wei, Wei; Zhang, Lei; Muqeet, Abdul; Hwang, Jiwon; Yang, Subin; Kang, JungHeum; Bae, Sung‐Ho; Kim, Yongwoo; Qu, Yanyun; Jeon, Geun-Woo; Choi, Jun‐Ho; Kim, Junhyuk; Lee, Jong-Seok; Marty, Steven; Marty, Eric; Xiong, Dongliang; Chen, Siang; Zha, Lin; Jiang, Joe‐Air; Gao, Xinbo; Lu, Wen; Wang, Haicheng; Bhaskara, Vineeth S.; Levinshtein, Alex; Tsogkas, Stavros; Jepson, Allan D.; Kong, Xiangzhen; Zhao, Tongtong; Zhao, Shanshan; Hrishikesh, P. S.; Puthussery, Densen; Jiji, C. V.; Nan, Nan; Liu, Shuai; Cai, Jie; Meng, Zibo; Ding, Jiaming; Ho, Chiu Man; Wang, Xuehui; Yan, Qing; Zhao, Yuzhi; Chen, Long; Sun, Lei; Wang, Wenhao; Liu, Zhenbing; Lan, Rushi; Umer, Rao Muhammad; Micheloni, Christian

doi:10.1007/978-3-030-67070-2_1

Cited by 59 publications

(33 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, a series of efficient networks with parameters in the range of 10M have been proposed for the efficient SR task [39]. We call these kinds of networks efficient super-resolution networks.…”

Section: Efficient Super-resolution Networkmentioning

confidence: 99%

“…Besides, model compression techniques such as knowledge distillation, channel pruning, and binary quantization have also been used to speed up the SR networks. Specifically, RFDN [27] applied channel pruning along with residual feature aggregation module to improve the IMDB efficiency, which is the winner solution of the AIM 2020 Challenge on Efficient Super-Resolution [39]. However, these methods aim to maximize PSNR between SR and HR, which tend to generate blurry results without high-frequency details.…”

Section: Efficient Super-resolution Networkmentioning

confidence: 99%

See 1 more Smart Citation

MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN

Cheng,

Zhao,

et al. 2021

Preprint

View full text Add to dashboard Cite

Generative adversarial networks (GANs) have promoted remarkable advances in single-image super-resolution (SR) by recovering photo-realistic images. However, high memory consumption of GAN-based SR (usually generators) causes performance degradation and more energy consumption, hindering the deployment of GAN-based SR into resource-constricted mobile devices. In this paper, we propose a novel compression framework Multi-scale Feature Aggregation Net based GAN (MFAGAN) for reducing the memory access cost of the generator. First, to overcome the memory explosion of dense connections, we utilize a memory-efficient multi-scale feature aggregation net as the generator. Second, for faster and more stable training, our method introduces the PatchGAN discriminator. Third, to balance the student discriminator and the compressed generator, we distill both the generator and the discriminator. Finally, we perform a hardware-aware neural architecture search (NAS) to find a specialized SubGenerator for the target mobile phone. Benefiting from these improvements, the proposed MFAGAN achieves up to 8.3× memory saving and 42.9× computation reduction, with only minor visual quality degradation, compared with ESRGAN. Empirical studies also show ∼70 milliseconds latency on Qualcomm Snapdragon 865 chipset.

show abstract

Section: Efficient Super-resolution Networkmentioning

confidence: 99%

Section: Efficient Super-resolution Networkmentioning

confidence: 99%

MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN

Cheng,

Zhao,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Lightweight and efficient CNN for SR tasks has been widely explored to suit mobile devices with an extremely small amount of parameters and computation [33,2,14,13,48,24,42,22]. In order to reduce the parameters, Kim et al [33] introduces recursive layers combined with residual schemes in the feature extraction stage.…”

Section: Lightweight Image Super-resolutionmentioning

confidence: 99%

Lightweight Image Super-Resolution with Hierarchical and Differentiable Neural Architecture Search

Huang¹,

Shen²,

He³

et al. 2021

Preprint

View full text Add to dashboard Cite

Single Image Super-Resolution (SISR) tasks have achieved significant performance with deep neural networks. However, the large number of parameters in CNNbased methods for SISR tasks require heavy computations. Although several efficient SISR models have been recently proposed, most are handcrafted and thus lack flexibility. In this work, we propose a novel differentiable Neural Architecture Search (NAS) approach on both the cell-level and network-level to search for lightweight SISR models. Specifically, the cell-level search space is designed based on an information distillation mechanism, focusing on the combinations of lightweight operations and aiming to build a more lightweight and accurate SR structure. The networklevel search space is designed to consider the feature connections among the cells and aims to find which information flow benefits the cell most to boost the performance. Unlike the existing Reinforcement Learning (RL) or Evolutionary Algorithm (EA) based NAS methods for SISR tasks, our search pipeline is fully differentiable, and the lightweight SISR models can be efficiently searched on both the cell-level and network-level jointly on a single GPU. Experiments show that our methods can achieve state-ofthe-art performance on the benchmark datasets in terms of PSNR, SSIM, and model complexity with merely 68G Multi-Adds for ×2 and 18G Multi-Adds for ×4 SR tasks. Code will be available at https://github.com/DawnHH/ DLSR-PyTorch.

show abstract

“…But large networks that contain several million parameters, for example EDSR [18] (combining ResNets and pixel-shuffle), are currently unable to reach the throughput needed for real-time applications on edge devices. Several so-called lightweight networks have been proposed for middle ground applications [38,15,19,35,3]. Typical lightweight networks use hundred of thousands parameters and are still beyond the capabilities of real-time applications on edge devices.…”

Section: Introductionmentioning

confidence: 99%

edge-SR: Super-Resolution For The Masses

Michelini¹,

Lu²,

Jiang³

2021

Preprint

View full text Add to dashboard Cite

Classic image scaling (e.g. bicubic) can be seen as one convolutional layer and a single upscaling filter. Its implementation is ubiquitous in all display devices and image processing software. In the last decade deep learning systems have been introduced for the task of image super-resolution (SR), using several convolutional layers and numerous filters. These methods have taken over the benchmarks of image quality for upscaling tasks. Would it be possible to replace classic upscalers with deep learning architectures on edge devices such as display panels, tablets, laptop computers, etc.? On one hand, the current trend in Edge-AI chips shows a promising future in this direction, with rapid development of hardware that can run deep-learning tasks efficiently. On the other hand, in image SR only few architectures have pushed the limit to extreme small sizes that can actually run on edge devices at realtime. We explore possible solutions to this problem with the aim to fill the gap between classic upscalers and small deep learning configurations. As a transition from classic to deep-learning upscaling we propose edge-SR (eSR), a set of one-layer architectures that use interpretable mechanisms to upscale images. Certainly, a one-layer architecture cannot reach the quality of deep learning systems. Nevertheless, we find that for high speed requirements, eSR becomes better at trading-off image quality and runtime performance. Filling the gap between classic and deeplearning architectures for image upscaling is critical for massive adoption of this technology. It is equally important to have an interpretable system that can reveal the inner strategies to solve this problem and guide us to future improvements and better understanding of larger networks.

show abstract

AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results

Cited by 59 publications

References 48 publications

MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN

MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN

Lightweight Image Super-Resolution with Hierarchical and Differentiable Neural Architecture Search

edge-SR: Super-Resolution For The Masses

Contact Info

Product

Resources

About