SinGAN-GIF: Learning a Generative Video Model from a Single GIF

Diverse Generation from a Single Video Made Possible

“…Comparison to other video generation methods: We further compare our method to recently published methods of diverse video generation from single video: HP-VAE-GAN [17] and SinGAN-GIF [3]. We show that our results are both qualitatively and quantitatively superior while reducing the runtime by a factor of ∼35, 000 (from 8 days training on one video to 18 seconds for new generated video).…”

“…In the video domain, the main goal is to generate novel instances but with the same content as in the reference video. [3] extends [36] to video in a straightforward manner, by employing 3D (space-time) convolutions. [17] Combines Variational Auto Encoders for the coarse scales, thus preventing mode collapse, with GANs for the fine scale to achieve improved quality.…”

“…Such methods (e.g., [1,47]) can typically produce high quality results only for a limited class of videos such as face expressions (e.g., [41]) or robot hand movement (e.g., [10]), and cannot be applied to arbitrary videos. On the other side of the spectrum, single video generative models have recently emerged [3,17]. These methods learn a generative video model from a single input video, in an unsupervised manner.…”

“…Right: A detailed comparison between nearest neighbors search time with PatchMatch (VGPNN) and with exhaustive search (GPNN*). GPNN* exceeds GPU memory at medium resolution (480p) with the original patch size(3,7,7). The dashed line with smaller patch size (3, 5, 5) is intended to show the quadratic trend with more data points.…”

Diverse Video Generation from a Single Video

“…On the other side of the spectrum are single-video GANs. These video generative models train on a single input video, learn its distribution of space-time patches, and are then able to generate a diversity of new videos with the same patch distribution [4,17]. However, these take very long time to train for each input video, making them applicable to only small spatial resolutions and to very short videos (typically, very few small frames).…”