Temporally Coherent Video Harmonization Using Adversarial Networks

Huang, Haozhi; Xu, Sen‐Zhe; Cai, Jin-Yi; Liu, Wei; Hu, Shi‐Min

doi:10.1109/tip.2019.2925550

Cited by 16 publications

(29 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An end-to-end deep convolutional neural network for image harmonization was proposed in [37], where the context and semantic information would improve harmonization. By contrast, Huang et al [38] engaged in video harmonization by designing a methodology based on the use of an end-to-end CNN that overcomes the flicker artifacts that occur when the image harmonization process is applied directly to videos. Moreover, Hou and Zhang proposed a method to recolor images starting from predefined color distributions.…”

Section: Recoloring Techniques For Images and Videosmentioning

confidence: 99%

Color Harmonization, Deharmonization and Balancing in Augmented Reality

2021

View full text Add to dashboard Cite

Color schemes play a crucial role in blending virtual objects with the real environment. Good color schemes improve user’s perception, which is of crucial importance for augmented reality. In this paper, we propose a set of novel methods based on the color harmonization methodology to recolor augmented reality content according to the real background. Three different strategies are proposed—harmonic, disharmonic, and balance—that allow for satisfying different needs in different settings depending on the application field. The first approach aims to harmonize the colors of virtual objects to make them consistent with the colors of the real background and reach a more pleasing effect to a human eye. The second approach, instead, can be adopted to generate a set of disharmonious colors with respect to real ones to be associated with the augmented virtual content to improve its distinctiveness from the real background. The third approach balances these goals by achieving a compromise between harmony and good visibility among virtual and real objects. Furthermore, the proposed re-coloring method is applied to three different case studies by adopting the three strategies to meet three different objectives, which are specific for each case study. Several parameters are calculated for each test, such as the covered area, the color distribution, and the set of generated colors. Results confirm the great potential of the proposed approaches to improve the AR visualization in different scenarios.

show abstract

Section: Recoloring Techniques For Images and Videosmentioning

confidence: 99%

Color Harmonization, Deharmonization and Balancing in Augmented Reality

2021

View full text Add to dashboard Cite

show abstract

“…However, composite videos are usually not realistic enough due to the appearance (e.g., illumination, color) incompatibility between foreground and background, which is caused by distinctive capture conditions (e.g., season, weather, time of the day) of foreground and background [4,3]. To address this issue, video harmonization [7] has been proposed to adjust the foreground appearance to make it compatible with the background, resulting in a more realistic composite video.…”

Section: Introductionmentioning

confidence: 99%

“…Recently, several deep learning based image harmonization methods [4,5,6,3,13,10] have been proposed. They changed the Although deep image harmonization methods have achieved remarkable success, directly applying them to video harmonization by harmonizing each frame separately will cause flickering artifacts [7], which largely downgrades the harmonization quality. Thus, it is imperative to design deep video harmonization method by taking temporal consistency into consideration.…”

Section: Introductionmentioning

confidence: 99%

“…Thus, it is imperative to design deep video harmonization method by taking temporal consistency into consideration. To the best of our knowledge, the only deep video harmonization method is [7], which proposed an end-to-end network to harmonize the composite frames while considering the temporal consistency between adjacent frames.…”

Section: Introductionmentioning

confidence: 99%

“…Training deep video harmonization network requires abundant pairs of composite videos and their ground-truth harmonized videos, but manually editing composite videos to obtain their harmonized videos is extremely tedious and expensive. Therefore, [7] adopted an inverse approach, that is, creating synthetic composite videos from real images. Particularly, they applied the traditional color transfer method [12] to the foreground of the real image to make it incompatible with the background, leading to the synthetic composite image.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

HYouTube: Video Harmonization Dataset

Lu¹,

Huang²,

Niu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Video composition aims to generate a composite video by combining the foreground of one video with the background of another video, but the inserted foreground may be incompatible with the background in terms of color and illumination. Video harmonization aims to adjust the foreground of a composite video to make it compatible with the background. So far, video harmonization has only received limited attention and there is no public dataset for video harmonization. In this work, we construct a new video harmonization dataset HYouTube by adjusting the foreground of real videos to create synthetic composite videos. Considering the domain gap between real composite videos and synthetic composite videos, we additionally create 100 real composite videos via copy-and-paste. Datasets are available at https://github.com/bcmi/Video-Harmonization-Dataset-HYouTube.

show abstract

Harmonizer: Learning to Perform White-Box Image and Video Harmonization

Sun

Zhu

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Temporally Coherent Video Harmonization Using Adversarial Networks

Cited by 16 publications

References 43 publications

Color Harmonization, Deharmonization and Balancing in Augmented Reality

Color Harmonization, Deharmonization and Balancing in Augmented Reality

HYouTube: Video Harmonization Dataset

Harmonizer: Learning to Perform White-Box Image and Video Harmonization

Contact Info

Product

Resources

About