Automatic Temporally Coherent Video Colorization

Thasarathan, Harrish; Nazeri, Kamyar; Ebrahimi, Mehran

doi:10.1109/crv.2019.00033

Cited by 21 publications

(46 citation statements)

References 24 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Results in Table 1 obtained from an existing baseline model trained on the Dragonball dataset with the previous ground truth frame as a condition in [15], still suffers from the flicker effect due to inconsistencies in color between subsequent frames. In addition, unfamiliar backgrounds and characters suffered the most as the model colored them differently for each frame.…”

Section: Discussionmentioning

confidence: 99%

Artist-Guided Semiautomatic Animation Colorization

Thasarathan

Ebrahimi

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

View full text Add to dashboard Cite

There is a delicate balance between automating repetitive work in creative domains while staying true to an artist's vision. The animation industry regularly outsources large animation workloads to foreign countries where labor is inexpensive and long hours are common. Automating part of this process can be incredibly useful for reducing costs and creating manageable workloads for major animation studios and outsourced artists. We present a method for automating line art colorization by keeping artists in the loop to successfully reduce this workload while staying true to an artist's vision. By incorporating color hints and temporal information to an adversarial image-to-image framework, we show that it is possible to meet the balance between automation and authenticity through artist's input to generate colored frames with temporal consistency.

show abstract

Section: Discussionmentioning

confidence: 99%

Artist-Guided Semiautomatic Animation Colorization

Thasarathan

Ebrahimi

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Network Architecture. We adopt the GAN based architecture for our HDR frame reconstruction proposed by [47]. The proposed generator network is based on encoder-decoder architecture, where the encoder first downsamples the image twice (𝐻 ×𝑊 → 𝐻 /4 ×𝑊 /4).…”

Section: Gan Based Hdr Frame Generationmentioning

confidence: 99%

“…The feature map is then passed through 8 Res-Blocks followed by two upsampling layers. Similar to [47], we use the instance norm layer. We warp a convolution layer, an instance-norm layer, and a ReLU activation layer into one basic unit.…”

Section: Gan Based Hdr Frame Generationmentioning

confidence: 99%

See 1 more Smart Citation

HDRVideo-GAN: Deep Generative HDR Video Reconstruction

Anand¹,

Harilal²,

Kumar‐Sinha³

et al. 2021

Preprint

View full text Add to dashboard Cite

High dynamic range (HDR) videos provide a more visually realistic experience than the standard low dynamic range (LDR) videos. Despite having significant progress in HDR imaging, it is still a challenging task to capture high-quality HDR video with a conventional off-the-shelf camera. Existing approaches rely entirely on using dense optical flow between the neighboring LDR sequences to reconstruct an HDR frame. However, they lead to inconsistencies in color and exposure over time when applied to alternating exposures with noisy frames. In this paper, we propose an end-to-end GAN-based framework for HDR video reconstruction from LDR sequences with alternating exposures. We first extract clean LDR frames from noisy LDR video with alternating exposures with a denoising network trained in a self-supervised setting. Using optical flow, we then align the neighboring alternating-exposure frames to a reference frame and then reconstruct high-quality HDR frames in a complete adversarial setting. To further improve the robustness and quality of generated frames, we incorporate temporal stabilitybased regularization term along with content and style-based losses in the cost function during the training procedure. Experimental results demonstrate that our framework achieves state-of-the-art performance and generates superior quality HDR frames of a video over the existing methods. CCS CONCEPTS• Computing methodologies → Computational photography; Image-based rendering.

show abstract

“…Most of these applications involve image processing. Although there have been some studies involving video processing, such as video generation [115], video colorization [116], [117], video inpainting [118], motion transfer [119], and facial animation synthesis [120]- [123], the research on video using GANs is limited. In addition, although GANs have been applied to the generation and synthesis of 3D models, such as 3D colorization [124], 3D face reconstruction [125], [126], 3D character animation [127], and 3D textured object generation [128], the results are far from perfect.…”

Section: B Future Opportunitiesmentioning

confidence: 99%

A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks

et al. 2020

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have achieved impressive results in various image synthesis tasks, and are becoming a hot topic in computer vision research because of the impressive performance they achieved in various applications. In this paper, we introduce the recent research on GANs in the field of image processing, including image synthesis, image generation, image semantic editing, image-to-image translation, image super-resolution, image inpainting, and cartoon generation. We analyze and summarize the methods used in these applications which have improved the generated results. Then, we discuss the challenges faced by GANs and introduce some methods to deal with these problems. We also preview some likely future research directions in the field of GANs, such as video generation, facial animation synthesis and 3D face reconstruction. The purpose of this review is to provide insights into the research on GANs and to present the various applications based on GANs in different scenarios. INDEX TERMS Generative adversarial networks, image synthesis, image-to-image translation, image editing, cartoon generation.

show abstract

Automatic Temporally Coherent Video Colorization

Cited by 21 publications

References 24 publications

Artist-Guided Semiautomatic Animation Colorization

Artist-Guided Semiautomatic Animation Colorization

HDRVideo-GAN: Deep Generative HDR Video Reconstruction

A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks

Contact Info

Product

Resources

About