Fast computation of seamless video loops

Liao, Jing; Finch, Mark; Hoppe, Hugues

doi:10.1145/2816795.2818061

Cited by 19 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that V M is used instead of V M loop if the looping process is not required. To make a motion loop V M loop from the nonperiodic sequence V M , various methods can be used [Liao et al 2015;Schödl et al 2000]. Among the several methods that we tested, simple cross-fading [Schödl et al 2000] worked relatively well for making plausible animations without significant discontinuities.…”

Section: Single-image Video Generationmentioning

confidence: 99%

Animating landscape

2019

View full text Add to dashboard Cite

Fig. 1. Given a single scenery image, our method predicts the motion (e.g., moving clouds) and appearance (e.g., time-varying colors) separately to generate a cyclic animation via self-supervised learning of time-lapse videos using our convolutional neural networks that infer backward flow fields (insets) and color transfer functions for converting the input image. The flow fields are visualized using the colormap shown in Figures 8 and 9. The output frame size is 1, 024 × 576. Please see the supplemental video for the resultant animations. Input photo: Pixabay/Pexel.com.Automatic generation of a high-quality video from a single image remains a challenging task despite the recent advances in deep generative models. This paper proposes a method that can create a high-resolution, long-term animation using convolutional neural networks (CNNs) from a single landscape image where we mainly focus on skies and waters. Our key observation is that the motion (e.g., moving clouds) and appearance (e.g., time-varying colors in the sky) in natural scenes have different time scales. We thus learn them separately and predict them with decoupled control while handling future uncertainty in both predictions by introducing latent codes. Unlike previous methods that infer output frames directly, our CNNs predict spatially-smooth intermediate data, i.e., for motion, flow fields for warping, and for appearance, color transfer maps, via self-supervised learning, i.e., without explicitly-provided ground truth. These intermediate data are applied not to each previous output frame, but to the input image only once for each output frame. This design is crucial to alleviate error accumulation in long-term predictions, which is the essential problem in previous recurrent approaches. The output frames can be looped like cinemagraph, and also be controlled directly by specifying latent codes or indirectly via visual annotations. We demonstrate the effectiveness of our method through comparisons with the state-of-the-arts on video prediction as well as appearance manipulation. Resultant videos, codes, and datasets will be available at

show abstract

Section: Single-image Video Generationmentioning

confidence: 99%

Animating landscape

2019

View full text Add to dashboard Cite

show abstract

“…Video Looping Liao et al [23] developed an automatic video-loop generation method that allows independently looping regions with separate periodicity and starting frames (optimized in a follow-up work [22]). The representation used in [22,23] conveys a wide spectrum of dynamism that a user can optionally select in the generated video loop. However, the output video loop is generated without any knowledge of the scene semantics; the dynamics of looping is computed based on continuity in appearance over space and time.…”

Section: Related Workmentioning

confidence: 99%

Personalized Cinemagraphs Using Semantic Understanding and Collaborative Learning

Joo

Joshi

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Cinemagraphs are a compelling way to convey dynamic aspects of a scene. In these media, dynamic and still elements are juxtaposed to create an artistic and narrative experience. Creating a high-quality, aesthetically pleasing cinemagraph requires isolating objects in a semantically meaningful way and then selecting good start times and looping periods for those objects to minimize visual artifacts (such a tearing). To achieve this, we present a new technique that uses object recognition and semantic segmentation as part of an optimization method to automatically create cinemagraphs from videos that are both visually appealing and semantically meaningful. Given a scene with multiple objects, there are many cinemagraphs one could create. Our method evaluates these multiple candidates and presents the best one, as determined by a model trained to predict human preferences in a collaborative way. We demonstrate the effectiveness of our approach with multiple results and a user study.

show abstract

“…Extensions for panoramic videos and stereo panoramic videos were proposed in [1] and [4] respectively. Liao et al [17,16] model motion on a per-pixel basis and segment the input video into spatio-temporal regions of similar motion automatically. Finally, Sevilla-Lara et al [29] focus on videos exhibiting camera motion which significantly increases the looping complexity and thus limit themselves to single dominant subjects.…”

Section: Video Looping and Animationmentioning

confidence: 99%

Responsive Action-based Video Synthesis

Ilisescu

Kanaci

Romagnoli

et al. 2017

Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

We propose technology to enable a new medium of expression, where video elements can be looped, merged, and triggered, interactively. Like audio, video is easy to sample from the real world but hard to segment into clean reusable elements. Reusing a video clip means non-linear editing and compositing with novel footage. The new context dictates how carefully a clip must be prepared, so our end-to-end approach enables previewing and easy iteration. We convert static-camera videos into loopable sequences, synthesizing them in response to simple end-user requests. This is hard because a) users want essentially semantic-level control over the synthesized video content, and b) automatic loop-finding is brittle and leaves users limited opportunity to work through problems. We propose a human-in-the-loop system where adding effort gives the user progressively more creative control. Artists help us evaluate how our trigger interfaces can be used for authoring of videos and video-performances.Comment: 10 pages, 12 figures, 1 table, accepted and published in Proceedings of the 2017 CHI Conference on Human Factors in Computing System

show abstract

Fast computation of seamless video loops

Cited by 19 publications

References 17 publications

Animating landscape

Animating landscape

Personalized Cinemagraphs Using Semantic Understanding and Collaborative Learning

Responsive Action-based Video Synthesis

Contact Info

Product

Resources

About