Recognizing and Localizing Individual Activities through Graph Matching

Abstract

In this paper we tackle the problem of detecting individual human actions in video sequences. While the most successful methods are based on local features, which proved that they can deal with changes in background, scale and illumination, most existing methods have two main shortcomings: first, they are mainly based on the individual power of spatio-temporal interest points (STIP), and therefore ignore the spatio-temporal relationships between them. Second, these methods mainly focus on direct classification techniques to classify the human activities, as opposed to detection and localization. In order to overcome these limitations, we propose a new approach, which is based on a graph matching algorithm for activity recognition. In contrast to most previous methods which classify entire video sequences, we design a video matching method from two sets of ST-points for human activity recognition. First, points are extracted, and a hyper graphs are constructed from them, i.e. graphs with edges involving more than 2 nodes (3 in our case). The activity recognition problem is then transformed into a problem of finding instances of model graphs in the scene graph. By matching local features instead of classifying entire sequences, our method is able to detect multiple different activities which occur simultaneously in a video sequence. Experiments on two standard datasets demonstrate that our method is comparable to the existing techniques on classification, and that it can, additionally, detect and localize activities.

Supervised Learning and Codebook Optimization for Bag-of-Words Models

“…Instead, the current state of the art focuses on sparse local features like interest points and space-time interest points, as the work cited above, or on motion segmentation through background subtraction [20][21][22], dense optical flow [23] or other holistic features [24,25], with possible hybrid methods [26][27][28] and classification through dense matching [29,30]. Fully taking into account spatial relationships through graph matching has recently been proposed [31], but this requires matching against several graph models per action class.…”

Spatial Joint features for 3D human skeletal action recognition system using spatial graph kernels

“…In [20], human motion in each frame is represented by a graph and matching similarity is calculated between training and test data. Graph based techniques, such as Adaptive Graph Kernels (AGK) in [13], Kuhn -Munkres graph matching algorithm [21] and Dynamic Programing (DP) [22] are used for 3D human motion matching. Graph kernels have received extensive appreciation from researchers on 3D continuous data [23].…”

Action recognition in videos

“…In [5] matching is done via temporally ordered local feature-graphs where each graph models spatial configuration of the features in a small temporal segment. In [11], we introduced a representation of actions as graphs built on ST keypoints and proximity information. Given a model keypoint set and a scene keypoint set, a possible solution of the problem is given through the values of a set variables x i , where a value of x i =j is interpreted as model point i being assigned to scene point j.…”