Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words

Abstract

We present a novel unsupervised learning method for human action categories. A video sequence is represented as a collection of spatial-temporal words by extracting space-time interest points. The algorithm automatically learns the probability distributions of the spatial-temporal words and the intermediate topics corresponding to human action categories. This is achieved by using latent topic models such as the probabilistic Latent Semantic Analysis (pLSA) model and Latent Dirichlet Allocation (LDA). Our approach can handle noisy feature points arisen from dynamic background and moving cameras due to the application of the probabilistic models. Given a novel video sequence, the algorithm can categorize and localize the human action(s) contained in the video. We test our algorithm on three challenging datasets: the KTH human motion dataset, the Weizmann human action dataset, and a recent dataset of figure skating actions. Our results reflect the promise of such a simple approach. In addition, our algorithm can recognize and localize multiple actions in long and complex video sequences containing multiple motions.

Socially intelligent surveillance and monitoring: Analysing social dimensions of physical space

“…A hierarchical notion of behavior, that enriches the serial action-based definition by adding different abstraction levels, is proposed in [18,49]. In [17], a completely novel concept of behavior is proposed, which consists of an ensemble of spatiotemporal feature points, deeply investigated in [50]. However, these structured definitions are based on the design of visual bricks (visual words) that do not carry any intuitive meanings.…”

Spatiotemporal pyramid representation for recognition of facial expressions and hand gestures

“…As noticed by [4] and observed from our experiments, the interest points detected by generalized space-time interest points detector from [16] are too sparse to build model for many complex activities. Therefore, we utilized the one from Dollar [4], which has been proven successful in [4,12,20]. Here we give a brief review of this method.…”

Discovering Video Clusters from Visual Features and Noisy Tags

“…HOG3D [2]) from video appearance, and then apply a standard clustering algorithm. For instance, Wang et al [3] cluster images strictly based on appearance, and Niebles et al [4] develop topic models based on video bag-of-words approaches. However, these methods are generally limited in performance due to the lack of semantics in low-level visual appearance.…”