Dynamic Scene Classification: Learning Motion Descriptors with Slow Features Analysis

Abstract

In this paper, we address the challenging problem of categorizing video sequences composed of dynamic natural scenes. Contrarily to previous methods that rely on handcrafted descriptors, we propose here to represent videos using unsupervised learning of motion features. Our method encompasses three main contributions: 1) Based on the Slow Feature Analysis principle, we introduce a learned local motion descriptor which represents the principal and more stable motion components of training videos. 2) We integrate our local motion feature into a global coding/pooling architecture in order to provide an effective signature for each video sequence. 3) We report state of the art classification performances on two challenging natural scenes data sets. In particular, an outstanding improvement of 11% in classification score is reached on a data set introduced in 2012.

Dynamic Scene Classification Using Spatial and Temporal Cues

“…From the confusion matrix, we observe that except for a scene like lightning, where there is a large variation in brightness level, the classification accuracy is high for all the scenes. Table 2 reports the comparison of our model with the state-of-the-art results: HOG [20,6], GIST [17,6], Chaos [24], SOE [6], and Slow Feature Analysis(SFA) [25]. Though there are variations in accuracies in individual categories, our model's average final accuracy appears similar to [25] model.…”

“…Table 2 reports the comparison of our model with the state-of-the-art results: HOG [20,6], GIST [17,6], Chaos [24], SOE [6], and Slow Feature Analysis(SFA) [25]. Though there are variations in accuracies in individual categories, our model's average final accuracy appears similar to [25] model. Experiment is repeated ten times randomly choosing the training set for computing the average accuracy for each category.…”

“…representative images from all of the datasets that are used for testing the proposed approach and are part of the UPenn dynamic scenes dataset [25]. We extract a frame from each video for applying spatial pyramid matching on the SIFT descriptor.…”

“…The clustering technique for pyramid matching can be performed in 2D image space and then extended to motion feature space [25]. We extract the local feature vectors in the corresponding feature spaces and quantize them into M types each of which is a code word in the codebook.…”

“…The other recent work on video scene classification involves a local descriptor based on Slow Feature Analysis (SFA), that represents stable and prime motion components of training video. These local descriptors are then integrated into global coding architecture for providing a model for each semantic category [25].…”