A Discriminative Model with Multiple Temporal Scales for Action Prediction

Abstract

The speed with which intelligent systems can react to an action depends on how soon it can be recognized. The ability to recognize ongoing actions is critical in many applications, for example, spotting criminal activity. It is challenging, since decisions have to be made based on partial videos of temporally incomplete action executions. In this paper, we propose a novel discriminative multi-scale model for predicting the action class from a partially observed video. The proposed model captures temporal dynamics of human actions by explicitly considering all the history of observed features as well as features in smaller temporal segments. We develop a new learning formulation, which elegantly captures the temporal evolution over time, and enforces the label consistency between segments and corresponding partial videos. Experimental results on two public datasets show that the proposed approach outperforms state-of-the-art action prediction methods.

Social LSTM: Human Trajectory Prediction in Crowded Spaces

“…A large body of work learns motion patterns through clustering trajectories [26,30,46,77]. More approaches can be found in [45,52,34,3,16,33]. Kitani et.…”

Early Action Prediction by Soft Regression

“…[11], [23], [40], [62], while less work has been reported on RGB-D sequences captured by low-cost depth cameras. In this work, we consider the early prediction of RGB-D action sequence and develop a real-time system for predicting human activities without any extra prior information about the progress level of on-going action when applied in practice.…”

“…In many real-world scenarios like surveillance, it would be more important to correctly predict an action before it is fully executed. Many efforts are on developing early action detectors or future action prediction systems [11], [20], [23], [33], [40], [51], [62]. For example, Hoai et al and Huang et al explored the application of max-margin learning in early event recognition and detection [11], [17].…”

Learning to Predict Human Behavior in Crowded Scenes

“…A large body of work learns motion patterns through clustering trajectories [27,28,29,30]. More approaches can be found in [31,32,33,34,35,36]. Kitani et.…”