NTIRE 2019 Challenge on Video Super-Resolution: Methods and Results

Nah, Seungjun; Timofte, Radu; Gu, Shuhang; Baik, Seung‐Chul; Hong, Seung Hwa; Moon, Gyeongsik; Son, Sanghyun; Lee, Kyoung Mu; Wang, Xintao; Chan, Kelvin C. K.; Yu, Ke; Dong, Chao; Loy, Chen Change; Yi, Fan; Yu, Jiahui; Liu, Ding; Huang, Thomas S.; Liu, Xiao; Li, Chao; He, Dongliang; Ding, Yukang; Wen, Shifeng; Porikli, Fatih; Kalarot, Ratheesh; Haris, Muhammad; Shakhnarovich, Greg; Ukita, Norimichi; Yi, Peng; Wang, Zhongyuan; Jiang, Kui; Jiang, Junjun; Ma, Jiayi; Dong, Hang; Zhang, Xinyi; Hu, Zhe; Kim, Kwanyoung; Kang, Dong Un; Chun, Se Young; Purohit, Kuldeep; Rajagopalan, A. N.; Tian, Yapeng; Zhang, Yulun; Fu, Yun; Xu, Chenliang; Tekalp, A. Murat; Yilmaz, M. Akin; Korkmaz, Cansu; Sharma, Manoj Kumar; Makwana, Megh; Badhwar, Anuj; Singh, Ajay Pratap; Upadhyay, Avinash; Mukhopadhyay, Rudrabha; Shukla, Ankit; Khanna, Dheeraj; Mandal, Atanu; Chaudhury, S. Raj; Miao, Si; Zhu, Yongxin; Huo, Xuexi

doi:10.1109/cvprw.2019.00250

Cited by 29 publications

(12 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We participated in all the four tracks in the video restoration and enhancement challenges [29,28], including video super-resolution (clean/blur) and video deblurring (clean/compression artifacts). Thanks to the effective alignment and fusion modules, our EDVR has won the champion in all the four challenging tracks, demonstrating the effectiveness and the generalizability of our method.…”

Section: Edvr (Ours) Dufmentioning

confidence: 99%

See 1 more Smart Citation

EDVR: Video Restoration With Enhanced Deformable Convolutional Networks

Wang

Chan

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

Self Cite

973

1,017

View full text Add to dashboard Cite

Video restoration tasks, including super-resolution, deblurring, etc, are drawing increasing attention in the computer vision community. A challenging benchmark named REDS is released in the NTIRE19 Challenge. This new benchmark challenges existing methods from two aspects:(1) how to align multiple frames given large motions, and (2) how to effectively fuse different frames with diverse motion and blur. In this work, we propose a novel Video Restoration framework with Enhanced Deformable convolutions, termed EDVR, to address these challenges. First, to handle large motions, we devise a Pyramid, Cascading and Deformable (PCD) alignment module, in which frame alignment is done at the feature level using deformable convolutions in a coarse-to-fine manner. Second, we propose a Temporal and Spatial Attention (TSA) fusion module, in which attention is applied both temporally and spatially, so as to emphasize important features for subsequent restoration. Thanks to these modules, our EDVR wins the champions and outperforms the second place by a large margin in all four tracks in the NTIRE19 video restoration and enhancement challenges. EDVR also demonstrates superior performance to state-of-the-art published methods on video super-resolution and deblurring. The code is available at https://github.com/xinntao/EDVR.

show abstract

Section: Edvr (Ours) Dufmentioning

confidence: 99%

“…We participated in all the four tracks in the NTIRE19 video restoration and enhancement challenges [29,28]. Quantitative results are presented in Table 5.…”

Section: Evaluation On Reds Datasetmentioning

confidence: 99%

EDVR: Video Restoration With Enhanced Deformable Convolutional Networks

Wang

Chan

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

Self Cite

973

1,017

View full text Add to dashboard Cite

show abstract

“…Since real-world SR is challenging due to unknown degradation and various noise [34,74,75,76,77,78,79,80], we also validate the effectiveness of our method for compressed inputs in Figure 19. Unlike previous experiments, FxSR and SRGAN [20] are re-trained using LR images compressed with JPEG quality factor 90, called FxSR-CA (compression artifacts) and SRGAN-CA.…”

Section: Compressed Lr Image Restorationmentioning

confidence: 69%

Flexible Style Image Super-Resolution using Conditional Objective

Park¹,

Moon²,

Cho³

2022

Preprint

View full text Add to dashboard Cite

Recent studies have significantly enhanced the performance of single-image super-resolution (SR) using convolutional neural networks (CNNs). While there can be many high-resolution (HR) solutions for a given input, most existing CNN-based methods do not explore alternative solutions during the inference. A typical approach to obtaining alternative SR results is to train multiple SR models with different loss weightings and exploit the combination of these models. Instead of using multiple models, we present a more efficient method to train a single adjustable SR model on various combinations of losses by taking advantage of multi-task learning. Specifically, we optimize an SR model with a conditional objective during training, where the objective is a weighted sum of multiple perceptual losses at different feature levels. The weights vary according to given conditions, and the set of weights is defined as a style controller. Also, we present an architecture appropriate for this training scheme, which is the Residual-in-Residual Dense Block equipped with spatial feature transformation layers. At the inference phase, our trained model can generate locally different outputs conditioned on the style control map. Extensive experiments show that the proposed SR model produces various desirable reconstructions without artifacts and yields comparable quantitative performance to stateof-the-art SR methods. Code and trained models will be available at https://github.com/seunghosnu/FxSR

show abstract

“…VFI has been extensively studied over the past decades to up-sample the frame rate of video content to match the ever increasing video monitor frame rates and produce smoother playback and slow-motion effects. Recently, CNNs have been leveraged to generate interpolated frames in real-time [31]. One of the better performing CNN-based systems is DAIN [25] which can interpolate frames in real-time.…”

Section: Video Frame Interpolationmentioning

confidence: 99%

Distributed Video Adaptive Block Compressive Sensing

Zammit,

Wassell

2021

Preprint

View full text Add to dashboard Cite

show abstract

NTIRE 2019 Challenge on Video Super-Resolution: Methods and Results

Cited by 29 publications

References 35 publications

EDVR: Video Restoration With Enhanced Deformable Convolutional Networks

EDVR: Video Restoration With Enhanced Deformable Convolutional Networks

Flexible Style Image Super-Resolution using Conditional Objective

Distributed Video Adaptive Block Compressive Sensing

Contact Info

Product

Resources

About