Motion Representation with Acceleration Images

Abstract

Information of time differentiation is extremely important cue for a motion representation. We have applied first-order differential velocity from a positional information, moreover we believe that second-order differential acceleration is also a significant feature in a motion representation. However, an acceleration image based on a typical optical flow includes motion noises. We have not employed the acceleration image because the noises are too strong to catch an effective motion feature in an image sequence. On one hand, the recent convolutional neural networks (CNN) are robust against input noises.In this paper, we employ acceleration-stream in addition to the spatial-and temporal-stream based on the two-stream CNN. We clearly show the effectiveness of adding the acceleration stream to the two-stream CNN.

Pedestrian Attributes and Activity Recognition Using Deep Learning: A Comprehensive Survey

“…Particularly, the three actions of "turning", " straight walking", and "crossing" are gaining fine-grained recognition due to their comparable form, changing pedestrian size, and inclusion in the same category as "walking. H. Kataoka et al [36] proposed a basic way for illustrating a change in a flow image using "acceleration images." These pictures should be significant since their representation differs from that of location (RGB) and velocity (flow) images.…”

Action Recognition With Coarse-to-Fine Deep Feature Integration and Asynchronous Fusion

“…Simonyan and Zisserman (Simonyan and Zisserman 2014) propose a twostream framework which uses two ConvNets to respectively extract features from two information streams (i.e., appearance and motion) and fuse them for recognition. Based on this framework, recent researches further improve the effectiveness of ConvNet features by including additional information sources (Shi et al 2017;Kataoka et al 2016), selecting spatial-temporal attention parts (Kar et al 2017;Sharma, Kiros, and Salakhutdinov 2015;Zhu et al 2016), or incorporating more proper temporal information (Wang et al 2016b;Wu et al 2015;Cherian et al 2017;Bilen et al 2016).…”

“…(Best viewed in color) less progress in action recognition due to the high complexity of video data. Some recent studies attempted to improve the deep feature representation of an action by including additional information sources (Duta et al 2017;Shi et al 2017;Kataoka et al 2016), selecting spatialtemporal attention parts (Kar et al 2017;Sharma, Kiros, and Salakhutdinov 2015;Zhu et al 2016), or incorporating more proper temporal information (Wang et al 2016b;Cherian et al 2017). However, since most of them focus on learning features to directly describe actions' individual action classes, they have limitations in precisely differentiating the ambiguity among action classes due to the large intraclass variations and subtle inter-class differences of actions.…”

“…Second, combining multiple information streams (such as two-stream ConvNets (Simonyan and Zisserman 2014)) has shown strong performance and thus has become a mainstream framework in action recognition. However, most existing works only focus on introducing more information streams (Shi et al 2017;Kataoka et al 2016) or strengthening the correlation among streams (Wang et al 2016b;Wu et al 2015;Sun et al 2017), while the asynchronous issue among different information streams is less studied.…”