TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution

Tian, Yapeng; Zhang, Yulun; Fu, Yun; Xu, Chenliang

doi:10.1109/cvpr42600.2020.00342

Cited by 432 publications

(327 citation statements)

References 29 publications

Supporting

Mentioning

327

Contrasting

Order By: Relevance

“…Instead of using optical flow, it uses deformable convolution to track the traceable points by a pyramid, cascading and deformable (PCD) module. Tian et al [51] proposed a time deformable alignment network (TDAN), which aligned adaptively at the feature level.…”

Section: Video Super-resolution (Vsr)mentioning

confidence: 99%

Video Super-Resolution Based on Generative Adversarial Network and Edge Enhancement

Wang

Teng

2021

Electronics

View full text Add to dashboard Cite

With the help of deep neural networks, video super-resolution (VSR) has made a huge breakthrough. However, these deep learning-based methods are rarely used in specific situations. In addition, training sets may not be suitable because many methods only assume that under ideal circumstances, low-resolution (LR) datasets are downgraded from high-resolution (HR) datasets in a fixed manner. In this paper, we proposed a model based on Generative Adversarial Network (GAN) and edge enhancement to perform super-resolution (SR) reconstruction for LR and blur videos, such as closed-circuit television (CCTV). The adversarial loss allows discriminators to be trained to distinguish between SR frames and ground truth (GT) frames, which is helpful to produce realistic and highly detailed results. The edge enhancement function uses the Laplacian edge module to perform edge enhancement on the intermediate result, which helps further improve the final results. In addition, we add the perceptual loss to the loss function to obtain a higher visual experience. At the same time, we also tried training network on different datasets. A large number of experiments show that our method has advantages in the Vid4 dataset and other LR videos.

show abstract

Section: Video Super-resolution (Vsr)mentioning

confidence: 99%

Video Super-Resolution Based on Generative Adversarial Network and Edge Enhancement

Wang

Teng

2021

Electronics

View full text Add to dashboard Cite

show abstract

“…(1) It would be highly challenging for them to estimate an accurate motion field, for instance, in the case of large-scale motions, and (2) even with a high-quality estimated optical flow, warped frames also suffer from severe artifacts caused by the motion compensation, which would be propagated into the final super-resolved frames. Therefore, some recent researches [40][41][42][43][44][45] tried to perform alignment in a non-ME&MC or implicit ME&MC manner. DUF [40] utilized learned dynamic up-sampling filters and residual images to super-resolved LR input frames.…”

Section: Alignmentmentioning

confidence: 99%

Video super‐resolution with non‐local alignment network

Zhou

Chen

Ding

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

Video super-resolution (VSR) aims at recovering high-resolution frames from their lowresolution counterparts. Over the past few years, deep neural networks have dominated the video super-resolution task because of its strong non-linear representational ability. To exploit temporal correlations, most deep neural networks have to face two challenges: (1) how to align consecutive frames containing motions, occlusions and blurring, and establish accurate temporal correspondences, (2) how to effectively fuse aligned frames and balance their contributions. In this work, a novel video super-resolution network, named NLVSR, is proposed to solve above problems in an efficient and effective manner. For alignment, a temporal-spatial non-local operation is employed to align each frame to the reference frame. Compared with existing alignment approaches, the proposed temporalspatial non-local operation is able to integrate the global information of each frame by a weighted sum, leading to a better performance in alignment. For fusion, an attentionbased progressive fusion framework was designed to integrate aligned frames gradually. To penalize the points with low-quality in aligned features, an attention mechanism was employed for a robust reconstruction. Experimental results demonstrate the superiority of the proposed network in terms of quantitative and qualitative evaluation, and surpasses other state-of-the-art methods by 0.33 dB at least. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

“…Recently, deformable convolution [57] has been received increasingly more attention to solve low-level vision tasks such as video super-resolution. EDVR [48] and TDAN [46] have already successfully implemented deformable convolution to temporally align reference frame and its neighboring frames which can let networks better utilize both spatial and temporal information to enhance the final results.…”

Section: Team-wvumentioning

confidence: 99%