Recognition of Transitional Action for Short-Term Action Prediction using Discriminative Temporal CNN Feature

Abstract

Transitional actions belong to a class between actions for short-term action prediction (see Figure 1). Early action recognition is necessary for producing action predictions in the early frames of an objective action. Earlier prediction in the initial frames of an objective action is desirable for early action recognition problems, but the solutions depend on the action itself. On one hand, within the setting of a shortterm action prediction, understanding a pending human action change is more natural if we have a firm grasp on transitional actions. In a traffic scene, short-term action predictions are particularly crucial for avoiding accidents between humans and vehicles. Figure 1 shows sequential actions that include Walk straight, Walk straight -cross, and cross. Where Walk straight and cross are conventional action definitions, our proposal adds a transitional action between actions (here Walk straight -cross) in order to provide a better action approach to predictions. Our proposed short-term predictions achieve earlier prediction than so-called early activity recognition, since they can recognize a dangerous cross action while it is transitional. Intuitively, the recognition difficulty arising from action and transitional action is that they tend to partially overlap each other. We believe that the use of a subtle motion descriptor (SMD) will allow us to identify sensitive differences between actions and transitional actions.In this paper, we address the recognition of transitional action for short-term action prediction. We also propose a discriminative temporal convolutional neural network (CNN) feature that can be used for recognizing transitional actions in order to overcome the difficulty of indistinguishable feature classification in transitional actions. To accomplish this, we employ an SMD that captures subtle differences between consecutive frames. Our paper contains two main contributions: (i) the definition of transitional action for short-term action prediction that achieves earlier prediction than early action recognition, and (ii) identifying CNN-based SMD to create a clear distinctions between action and transitional action. The feature is simply updated from a spatio-temporal CNN feature Pooled Time Series (PoT) proposed in [1].Our CNN-based SMD demonstrated the best rate of success on three different trial datasets. Even when using the shortest (3-frame) feature accumulation for recognition tuning, we confirmed outstanding results with 85.78% (NTSEL), 69.77% (UTKinect), and 49.93% (Watch-n-Patch) on the three different datasets.[1] M. S. Ryoo, B. Rothrock, and L. Matthies. Pooled motion features for first-person videos. CVPR, 2015.

A New Bayesian Modeling for 3D Human-Object Action Recognition

“…To handle the limited size of the aforementioned datasets, H. Kataoka et al [11] While achieving undisputed performances when training data is sufficiently diverse and large, end-to-end deeplearning action recognition frameworks suffer from a lack of interpretability of their features [26]. When a CNN does not recognize an action, it is difficult to know whether the issue is related to the implicit inference of the object detection and/or related to human pose.…”

“…We show that the additional information about objects configuration and their nature enhance action models, thus action recognition performances. The approach KMHIS [11] relies on motion pattern recognition in 2D through differential images. The significant improvement is also due to our 3D modeling of the scene being more robust to changes in camera viewpoints present in this dataset.…”

Drive Video Analysis for the Detection of Traffic Near-Miss Incidents

“…As for semantic segmentation, we can now obtain knowledge about dense connections with graphical models and multi-scale CNN [10], [11]. The usage of spatiotemporal analysis successfully predicts a future situation of pedestrians [12], [13].…”

“…Pooled time series (PoT) [33] and subtle motion descriptor (SMD) [12]. The settings for both were based on [12], which adjusted the parameters for short-term recognition from the understanding of a long-term event [33]. We used a 10-frame accumulation to ensure high accuracy in near-miss recognition and detection.…”

Research on rehabilitation training bed with action prediction based on NARX neural network

“…The common way is to pretreat the data after obtaining the relevant information, extract the data features which can exactly represent the action, and adopt appropriate recognition algorithm to distinguish different intentions. The analysis methods mainly focus on principal component analysis, wavelet transform, Kalman filter, K nearest neighbor algorithm, FFT transform, support vector machine, and so on …”