Action bank: A high-level representation of activity in video

Abstract

Activity recognition in video is dominated by low-and mid-level features, and while demonstrably capable, by nature, these features carry little semantic meaning. Inspired by the recent object bank approach to image representation, we present Action Bank, a new high-level representation of video. Action bank is comprised of many individual action detectors sampled broadly in semantic space as well as viewpoint space. Our representation is constructed to be semantically rich and even when paired with simple linear SVM classifiers is capable of highly discriminative performance. We have tested action bank on four major activity recognition benchmarks. In all cases, our performance is better than the state of the art, namely 98.2% on KTH (better by 3.3%), 95.0% on UCF Sports (better by 3.7%), 57.9% on UCF50 (baseline is 47.9%), and 26.9% on HMDB51 (baseline is 23.2%). Furthermore, when we analyze the classifiers, we find strong transfer of semantics from the constituent action detectors to the bank classifier.

Modeling Interactions between Low-Level and High-Level Features for Human Action Recognition

“…Since Sadanand and Corso [5] use richer representation for videos, they report a recognition accuracy of 98% on KTH actions dataset. However, there are three main advantages of our method compared to [5].…”

“…However, there are three main advantages of our method compared to [5]. First, our method automatically reports the high-level features (camera position features) and provides a semantic description for videos.…”

“…for i ∈ P do 5 Loss P (h) = 1 − x T i w(h i − h), 6 where w ≡ arg min w h ∈ S P j 7 computeĥ = arg max h∈H Loss P (h) …”

“…However, ST features are limited in capturing semantics information and often yield a representation with inadequate discriminative power for larger, more complex datasets. Consequently, high-level features are introduced to enrich ST features with semantic information and give a robust representation for human action recognition [4], [5].…”

Instantaneous threat detection based on a semantic representation of activities, zones and trajectories

“…However, they are not directly associated with persons, zones in the scene, a person's activities, and what happens during a person's trajectory. Such associations are not trivial: many of the well-performing methods consider the low-level features in the whole video [3], the whole scene [4], or in sub-volumes without making explicit associations [5]. Recent attempts for complex behaviours in complex scenes have not been successful yet [6], although reasonable performance have been reported for simple activities [7].…”