MoCoGAN: Decomposing Motion and Content for Video Generation

Tulyakov, Sergey; Li, Mingyu; Yang, Xiaodong; Kautz, Jan

doi:10.48550/arxiv.1707.04993

Cited by 48 publications

(87 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work is closely related to the research in video synthesis, which explores future frame prediction [18,1,3], future clip prediction [37,12], and conditioned video generation [33,31,2,28,23]. In one of the early attempts on video prediction, the authors in [18] proposed a GAN based approach for next-frame prediction, where they explore different types of loss functions along with adversarial loss.…”

Section: Related Workmentioning

confidence: 99%

“…In one of the early attempts on video prediction, the authors in [18] proposed a GAN based approach for next-frame prediction, where they explore different types of loss functions along with adversarial loss. Similarly, the authors in [33,2,37,12,31,23,28] explored the GAN framework for video synthesis, where they focus on future frame prediction and conditional video generation. The authors in [1] recently proposed a variational latent space learning framework for video synthesis, and similarly the authors in [3] also explored a recurrent approach for next frame prediction.…”

Section: Related Workmentioning

confidence: 99%

“…We have recently seen some works for video synthesis which make use of adversarial loss in the GAN framework [33,23,31]. We compare the quality of the generated videos with these works in terms of inception score [23].…”

Section: D1 Inception Scorementioning

confidence: 99%

“…In recent years, we have seen some success in the research on video synthesis. The proposed methods are mainly focused on future frame prediction [18,1,3], future clip prediction [33,37,12], or conditioned video generation [31,2,28,23]. The future frame/clip prediction methods can synthesize frames from near future in the video with some realism.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Time-Aware and View-Aware Video Rendering for Unsupervised Representation Learning

Vyas,

Rawat,

Shah

2018

Preprint

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: D1 Inception Scorementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Time-Aware and View-Aware Video Rendering for Unsupervised Representation Learning

Vyas,

Rawat,

Shah

2018

Preprint

View full text Add to dashboard Cite

“…The core of the training of GANs is a min-max game in which two neural networks (generator and discriminator) compete with each other: the generator tries to trick the discriminator/classifier into classifying its generated synthetic/fake data as true. Various applications have benefited from the utilization of GANs, e.g., video prediction, object generation, photo super resolution (see [13,26] and the references therein).…”

Section: Introductionmentioning

confidence: 99%

MR-GAN: Manifold Regularized Generative Adversarial Networks

Li,

Kailkhura,

Anirudh

et al. 2018

Preprint

View full text Add to dashboard Cite

Despite the growing interest in generative adversarial networks (GANs), training GANs remains a challenging problem, both from a theoretical and a practical standpoint. To address this challenge, in this paper, we propose a novel way to exploit the unique geometry of the real data, especially the manifold information. More specifically, we design a method to regularize GAN training by adding an additional regularization term referred to as manifold regularizer. The manifold regularizer forces the generator to respect the unique geometry of the real data manifold and generate high quality data. Furthermore, we theoretically prove that the addition of this regularization term in any class of GANs including DCGAN and Wasserstein GAN leads to improved performance in terms of generalization, existence of equilibrium, and stability. Preliminary experiments show that the proposed manifold regularization helps in avoiding mode collapse and leads to stable training.

show abstract

Factorised Spatial Representation Learning: Application in Semi-supervised Myocardial Segmentation

Chartsias

Joyce

Papanastasiou³

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The success and generalisation of deep learning algorithms heavily depend on learning good feature representations. In medical imaging this entails representing anatomical information, as well as properties related to the specific imaging setting. Anatomical information is required to perform further analysis, whereas imaging information is key to disentangle scanner variability and potential artefacts. The ability to factorise these would allow for training algorithms only on the relevant information according to the task. To date, such factorisation has not been attempted. In this paper, we propose a methodology of latent space factorisation relying on the cycle-consistency principle. As an example application, we consider cardiac MR segmentation, where we separate information related to the myocardium from other features related to imaging and surrounding substructures. We demonstrate the proposed method's utility in a semi-supervised setting: we use very few labelled images together with many unlabelled images to train a myocardium segmentation neural network. Specifically, we achieve comparable performance to fully supervised networks using a fraction of labelled images in experiments on ACDC and a dataset from Edinburgh Imaging Facility QMRI. Code will be made available at

show abstract

MoCoGAN: Decomposing Motion and Content for Video Generation

Cited by 48 publications

References 0 publications

Time-Aware and View-Aware Video Rendering for Unsupervised Representation Learning

Time-Aware and View-Aware Video Rendering for Unsupervised Representation Learning

MR-GAN: Manifold Regularized Generative Adversarial Networks

Factorised Spatial Representation Learning: Application in Semi-supervised Myocardial Segmentation

Contact Info

Product

Resources

About