On Diverse Asynchronous Activity Anticipation

Zhao, He; Wildes, Richard P.

doi:10.1007/978-3-030-58526-6_46

Cited by 15 publications

(22 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in the case of applying a GAN on discrete data, the discrete and stochastic fashion of the generator's output hampers direct differentiability, and has therefore been a suitable candidate for Gumbel-based gradient estimators. Applications of discrete GANs that have leveraged these Gumbel-based gradient estimators include (among others) text generation [50], [51], [52], [53], [54], [55], [56], fake user data creation for recommender systems [57], and action prediction [58]. Discrete GANs have also been combined with knowledge distillation frameworks.…”

Section: Discrete Gansmentioning

confidence: 99%

“…Conversely, when GS is used in discrete GANs, entropy of the generator's (output) distribution is often maximized rather than minimized, in order to stabilize optimization and prevent mode collapse (which is the case when the generator outputs non-diverse samples) [40], [54], [96]. To deal with this diversity-quality dilemma in GANs, also distance regularization in latent space has been leveraged [52], [58], promoting closeness of embeddings, and therefore (indirectly) influencing the logits predicted by the generator.…”

Section: Initialization and Regularizationmentioning

confidence: 99%

See 1 more Smart Citation

A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning

Huijben¹,

Kool²,

Paulus³

et al. 2021

Preprint

View full text Add to dashboard Cite

The Gumbel-max trick is a method to draw a sample from a categorical distribution, given by its unnormalized (log-)probabilities. Over the past years, the machine learning community has proposed several extensions of this trick to facilitate, e.g., drawing multiple samples, sampling from structured domains, or gradient estimation for error backpropagation in neural network optimization. The goal of this survey article is to present background about the Gumbel-max trick, and to provide a structured overview of its extensions to ease algorithm selection. Moreover, it presents a comprehensive outline of (machine learning) literature in which Gumbel-based algorithms have been leveraged, reviews commonly-made design choices, and sketches a future perspective.

show abstract

Section: Discrete Gansmentioning

confidence: 99%

Section: Initialization and Regularizationmentioning

confidence: 99%

A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning

Huijben¹,

Kool²,

Paulus³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Discrete time format matches the original scheme naturally while continuous ones incur unnecessary decimals. Recent work succeeds the discrete time scheme and trains a conditional GAN model based on Gumbel discrete sampling to jointly enhance the accuracy and diversity in both future action semantics and times [183]; see Figure 3.9. Previous efforts tended to be challenged in producing realistic (accurate) and diverse predictions.…”

Section: Add_milk Crack_egg Add_buttermentioning

confidence: 99%

“…Some work aims at learning the mapping from high-level information, extracted from raw observations, to the future. These efforts usually depend on extracted/estimated activity labels and segmented temporal duration as input [1,2,61,105,183].…”

Section: Add_milk Crack_egg Add_buttermentioning

confidence: 99%

See 1 more Smart Citation

Review of Video Predictive Understanding: Early Action Recognition and Future Action Prediction

Zhao¹,

Wildes²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Video predictive understanding encompasses a wide range of efforts that are concerned with the anticipation of the unobserved future from the current as well as historical video observations. Action prediction is a major sub-area of video predictive understanding and is the focus of this review. This sub-area has two major subdivisions: early action recognition and future action prediction. Early action recognition is concerned with recognizing an ongoing action as soon as possible. Future action prediction is concerned with the anticipation of actions that follow those previously observed. In either case, the causal relationship between the past, current and potential future information is the main focus.Various mathematical tools such as Markov Chains, Gaussian Processes, Auto-Regressive modeling and Bayesian recursive filtering are widely adopted jointly with computer vision techniques for these two tasks.However, these approaches face challenges such as the curse of dimensionality, poor generalization and constraints from domain specific knowledge. Recently, structures that rely on deep convolutional neural networks and recurrent neural networks have been extensively proposed for improving performance of existing vision tasks, in general, and action prediction tasks, in particular. However, they have their own shortcomings, e.g., reliance on massive training data and lack of strong theoretical underpinnings. In this survey, we start by introducing the major sub-areas of the broad area of video predictive understanding, which recently have received intensive attention and proven to have practical value. Next, a thorough review of various early action recognition and future action prediction algorithms are provided with suitably organized divisions. Finally, we conclude our discussion with future research directions.

show abstract