Bringing Old Films Back to Life

Wan, Ziyu; Zhang, Bo; Chen, Dongdong; Liao, Jing

doi:10.48550/arxiv.2203.17276

Cited by 2 publications

(3 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It becomes a recurrent model when N = 1 or a transformer model when N = T . This is fundamentally different from previous methods that adopt transformer blocks to replace CNN blocks within a recurrent architecture [77,43]. It is also different from existing attempts in natural language processing [81,33].…”

Section: Recurrent Feature Refinementmentioning

confidence: 69%

See 1 more Smart Citation

Recurrent Video Restoration Transformer with Guided Deformable Attention

Liang¹,

Yi²,

Xiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Video restoration aims at restoring multiple high-quality frames from multiple lowquality frames. Existing video restoration methods generally fall into two extreme cases, i.e., they either restore all frames in parallel or restore the video frame by frame in a recurrent way, which would result in different merits and drawbacks. Typically, the former has the advantage of temporal information fusion. However, it suffers from large model size and intensive memory consumption; the latter has a relatively small model size as it shares parameters across frames; however, it lacks long-range dependency modeling ability and parallelizability. In this paper, we attempt to integrate the advantages of the two cases by proposing a recurrent video restoration transformer, namely RVRT. RVRT processes local neighboring frames in parallel within a globally recurrent framework which can achieve a good trade-off between model size, effectiveness, and efficiency. Specifically, RVRT divides the video into multiple clips and uses the previously inferred clip feature to estimate the subsequent clip feature. Within each clip, different frame features are jointly updated with implicit feature aggregation. Across different clips, the guided deformable attention is designed for clip-to-clip alignment, which predicts multiple relevant locations from the whole inferred clip and aggregates their features by the attention mechanism. Extensive experiments on video super-resolution, deblurring, and denoising show that the proposed RVRT achieves state-of-the-art performance on benchmark datasets with balanced model size, testing memory and runtime. The codes are available at https://github.com/JingyunLiang/RVRT.Preprint. Under review.

show abstract

Section: Recurrent Feature Refinementmentioning

confidence: 69%

“…It aims to restore a clear and sharp high-quality video from a degraded (e.g., downsampled, blurred, or noisy) low-quality video [79,9,4,37]. It has wide applications in live streaming [96], video surveillance [48], old film restoration [77], and more.…”

Section: Introductionmentioning

confidence: 99%

Recurrent Video Restoration Transformer with Guided Deformable Attention

Liang¹,

Yi²,

Xiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…These models are refined through extensive training regimes on diverse datasets, enabling them to master intricate color mappings, thereby enhancing the colorization process. Scholars such as Zhang [7], Wan [28], Chen [29], and Iizuka [30] have predominantly embraced this technique for imparting color to grayscale video content, and it is this very technique that forms the crux of the present paper's investigation.…”

Section: Video Colorizationmentioning

confidence: 99%

Video Colorization Based on Variational Autoencoder

Zhang,

Hong,

Liu

et al. 2024

Electronics

View full text Add to dashboard Cite

This paper introduces a variational autoencoder network designed for video colorization using reference images, addressing the challenge of colorizing black-and-white videos. Although recent techniques perform well in some scenarios, they often struggle with color inconsistencies and artifacts in videos that feature complex scenes and long durations. To tackle this, we propose a variational autoencoder framework that incorporates spatio-temporal information for efficient video colorization. To improve temporal consistency, we unify semantic correspondence with color propagation, allowing for simultaneous guidance in colorizing grayscale video frames. Additionally, the variational autoencoder learns spatio-temporal feature representations by mapping video frames into a latent space through an encoder network. The decoder network then transforms these latent features back into color images. Compared to traditional coloring methods, our approach accurately captures temporal relationships between video frames, providing precise colorization while ensuring video consistency. To further enhance video quality, we apply a specialized loss function that constrains the generated output, ensuring that the colorized video remains spatio-temporally consistent and natural. Experimental results demonstrate that our method significantly improves the video colorization process.

show abstract

Bringing Old Films Back to Life

Cited by 2 publications

References 47 publications

Recurrent Video Restoration Transformer with Guided Deformable Attention

Recurrent Video Restoration Transformer with Guided Deformable Attention

Video Colorization Based on Variational Autoencoder

Contact Info

Product

Resources

About