Darwintrees for Action Recognition

Abstract

We propose a novel mid-level representation for action/activity recognition on RGB videos. We model the evolution of improved dense trajectory features not only for the entire video sequence, but also on subparts of the video. Subparts are obtained using a spectral divisive clustering that yields an unordered binary tree decomposing the entire cloud of trajectories of a sequence. We then compute videodarwin on video subparts, exploiting more finegrained temporal information and reducing the sensitivity of the standard time varying mean strategy of videodarwin. After decomposition, we model the evolution of features through both frames of subparts and descending/ascending paths in tree branches. We refer to these mid-level representations as node-darwintree and branch-darwintree respectively. For the final classification, we construct a kernel representation for both mid-level and holistic videodarwin representations. Our approach achieves better performance than standard videodarwin and defines the current state-of-the-art on UCF-Sports and Highfive action recognition datasets.

Action recognition based on motion of oriented magnitude patterns and feature selection

“…First, our system obtains better results in comparison to local temporal methods with the similar experimental settings. Second, even though our method does not achieve the same rates as in [13] (84.2%), [45] (87.3%), and [18] (91.5%), one advantage of our method is the use of simple local feature extraction, compared to the more computationally expensive trajectories or complex deep learning approaches [1, 2].…”

“…memory usage and disk space), and can lead to over-fitting due to a large number of parameters. In applications with constraints of data, the shallow learning approaches which are based on hand-crafted features [13][14][15][16][17][18] seem useful thanks to their remarkable results and understandability (we would like to distinguish the deep and shallow systems. The first ones often consist of many layers, for example, 16 layers in VGG16Net, while the second ones contain less layers).…”

“…Leave-one-out cross-validation is recommended as in [19] to test on UCF Sports. This scheme takes out one sample video for testing and uses all of the remaining videos of an action class for Harris3D + HOG [31] 71.4 Harris3D + HOF [31] 75.4 Harris3D + HOG/HOF [31] 78.1 Dense + HOG/HOF [31] 81.6 Hessian + HOG/HOF [31] 79.3 Gabor + HOG/HOF [55] 77.7 Kovashka and Grauman [45] 87.3 Le et al [2] 86.5 ST-SIFT + HOG3D [56] 80.5 MBH + Dense trajectories [13] 84.2 VideoDarwin + Dense trajectories [18] 91. 29.2 Sadanand and Corso [48] 26.9 ours 3 (MOMP + PCA + VLAD + linear SVM) 31.5 Misra et al [60] 30.6 Fernando et al [57] 65.0 Varol et al [58] 67.2 Feichtenhofer et al [59] 72.2 training.…”

“…Some of the best performing deep learning-based methods are still based on handcrafted features [3,7]. Therefore, the shallow learning approaches, which are based on hand-crafted features [11][12][13][14][15][16][17][18], are still used thanks to their comprehensibility and efficiency, notably in applications where there are constraints of small data sets and lowpower resources.…”