FutureGAN: Anticipating the Future Frames of Video Sequences using Spatio-Temporal 3d Convolutions in Progressively Growing GANs

Aigner, Sandra; Körner, Marco

doi:10.48550/arxiv.1810.01325

Cited by 16 publications

(24 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Video prediction is a challenging task due to the complex task of modelling both the content and motion in videos. To this extent several studies have been carried out to perform video prediction using GAN-based training ( [114], [164], [2], [66], [104], [166], [108]). Mathieu et al [114] used multi-scale architecture for future frame prediction.…”

Section: Video Predictionmentioning

confidence: 99%

A review of Generative Adversarial Networks (GANs) and its applications in a wide variety of disciplines -- From Medical to Remote Sensing

Dash¹,

Ye²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

We look into Generative Adversarial Network (GAN), its prevalent variants and applications in a number of sectors. GANs combine two neural networks that compete against one another using zero-sum game theory, allowing them to create much crisper and discrete outputs. GANs can be used to perform image processing, video generation and prediction, among other computer vision applications.GANs can also be utilised for a variety of science-related activities, including protein engineering, astronomical data processing, remote sensing image dehazing, and crystal structure synthesis. Other notable fields where GANs have made gains include finance, marketing, fashion design, sports, and music. Therefore in this article we provide a comprehensive overview of the applications of GANs in a wide variety of disciplines. We first cover the theory supporting GAN, GAN variants, and the metrics to evaluate GANs.Then we present how GAN and its variants can be applied in twelve domains, ranging from STEM fields, such as astronomy and biology, to business fields, such as marketing and finance, and to arts, such as music. As a result, researchers from other fields may grasp how GANs work and apply them to their own study. To the best of our knowledge, this article provides the most comprehensive survey of GAN's applications in different field. CCS Concepts: • General and reference → Surveys and overviews; • Computing methodologies → Computer vision; Neural networks; • Theory of computation → Machine learning theory.

show abstract

Section: Video Predictionmentioning

confidence: 99%

A review of Generative Adversarial Networks (GANs) and its applications in a wide variety of disciplines -- From Medical to Remote Sensing

Dash¹,

Ye²,

Wang³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Predicting the whereabouts of an emerging object inherits predicting the future environment itself. Predicting the future environment was addressed by predicting future frames [55,52,51,1,28,32,61] and future semantic segmentation [34,25,54,33,8,6]. These methods can only hallucinate new objects in the scene in a photorealistic way, but none of them explicitly predicts the structure where new objects can actually emerge.…”

Section: Related Workmentioning

confidence: 99%

“…In this paper (1) we propose a future localization framework in egocentric view by transferring the work by Makansi et al [36] from bird's-eye view to egocentric observations, where multimodality is even more difficult to capture. Thus, (2) we propose to compute a reachability prior as intermediate result, which serves as attention to prevent forgetting rare modes, and which can be used to efficiently propagate scene priors into the future taking into account the egomotion.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Future Localization and Emergence Prediction for Objects in Egocentric View with a Reachability Prior

Makansi¹,

Çiçek²,

Buchicchio³

et al. 2020

Preprint

View full text Add to dashboard Cite

In this paper, we investigate the problem of anticipating future dynamics, particularly the future location of other vehicles and pedestrians, in the view of a moving vehicle. We approach two fundamental challenges: (1) the partial visibility due to the egocentric view with a single RGB camera and considerable field-of-view change due to the egomotion of the vehicle; (2) the multimodality of the distribution of future states. In contrast to many previous works, we do not assume structural knowledge from maps. We rather estimate a reachability prior for certain classes of objects from the semantic map of the present image and propagate it into the future using the planned egomotion. Experiments show that the reachability prior combined with multi-hypotheses learning improves multimodal prediction of the future location of tracked objects and, for the first time, the emergence of new objects. We also demonstrate promising zero-shot transfer to unseen datasets.

show abstract

“…Video Generation. Many approaches use GANs [1,34,36] or adversarial loss during training for generating videos [1,21,23,25,31,36,37]. Vondrick et al [36] use a generator with two pathways to predict the foreground and background, and a discriminator to distinguish a video as real or fake.…”

Section: Related Workmentioning

confidence: 99%

Point-to-Point Video Generation

Wang

Cheng

Lin

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

FutureGAN: Anticipating the Future Frames of Video Sequences using Spatio-Temporal 3d Convolutions in Progressively Growing GANs

Cited by 16 publications

References 0 publications

A review of Generative Adversarial Networks (GANs) and its applications in a wide variety of disciplines -- From Medical to Remote Sensing

A review of Generative Adversarial Networks (GANs) and its applications in a wide variety of disciplines -- From Medical to Remote Sensing

Multimodal Future Localization and Emergence Prediction for Objects in Egocentric View with a Reachability Prior

Point-to-Point Video Generation

Contact Info

Product

Resources

About