Traffic Command Gesture Recognition for Virtual Urban Scenes Based on a Spatiotemporal Convolution Neural Network

Abstract

Intelligent recognition of traffic police command gestures increases authenticity and interactivity in virtual urban scenes. To actualize real-time traffic gesture recognition, a novel spatiotemporal convolution neural network (ST-CNN) model is presented. We utilized Kinect 2.0 to construct a traffic police command gesture skeleton (TPCGS) dataset collected from 10 volunteers. Subsequently, convolution operations on the locational change of each skeletal point were performed to extract temporal features, analyze the relative positions of skeletal points, and extract spatial features. After temporal and spatial features based on the three-dimensional positional information of traffic police skeleton points were extracted, the ST-CNN model classified positional information into eight types of Chinese traffic police gestures. The test accuracy of the ST-CNN model was 96.67%. In addition, a virtual urban traffic scene in which real-time command tests were carried out was set up, and a real-time test accuracy rate of 93.0% was achieved. The proposed ST-CNN model ensured a high level of accuracy and robustness. The ST-CNN model recognized traffic command gestures, and such recognition was found to control vehicles in virtual traffic environments, which enriches the interactive mode of the virtual city scene. Traffic command gesture recognition contributes to smart city construction.

Systematic literature review of hand gestures used in human computer interaction interfaces

“… Interaction with a display/projection (Bolt, 1980, Choi et al, 2007, Foehrenbach et al, 2009, Beyer and Meier, 2011, Asadzadeh et al, 2012, Cauchard et al, 2012, Xie and Xu, 2013, Rossol et al, 2014, Saxen et al, 2014, Adeen et al, 2015, Braun et al, 2017, Osti et al, 2017, Dondi et al, 2018, Ma et al, 2018.  Interaction with augmented reality (Reifinger et al, 2007, Lu et al, 2012, Bai et al, 2013, Hürst and van Wezel, 2013, Gangman and Yen, 2014, Adhikarla et al, 2015, Hernoux and Christmann, 2015, Shim et al, 2016, Saxen et al, 2014, Kim and Lee, 2016, Memo and Zanuttigh, 2018.…”

Automatic Discovery of Railway Train Driving Modes Using Unsupervised Deep Learning

“…A set of discriminative features plays a vital role in finding modes hidden in transportation trajectory data. In existing studies, supervised features (i.e., artificially designed features) such as global statistical features [14,20,[27][28][29][30][31][32], local features [19,33], time-domain features [20], frequency-domain features [20,27], and specific features [28,30,[34][35][36][37] have been extracted. Zheng et al [2] proposed a type of feature that can reveal the differences between railway driving behaviors, therein being designed for a railway mode identification context.…”

Identifying Modes of Driving Railway Trains from GPS Trajectory Data: An Ensemble Classifier-Based Approach

“…In existing studies, global features [19,21,[25][26][27][28][29], local features [12,30], time-domain features [26], frequency-domain features [25,26] and specific features [19,28,[31][32][33] were extracted through corresponding methods. Among these features, global features focus on describing whole characteristics of trajectory parameters (e.g., speed, acceleration, etc.)…”

“…As to classifiers, many studies have introduced different classifiers into transportation mode detection, such as Decision Tree [14,19,24,26,34], K-Nearest-Neighbor (KNN) [21,34,35], Support Vector Machines (SVM) [12,14,21,34], Artificial Neural Network (ANN) [20,28,33], Fuzzy Logic [13,28], Hidden Markov Models [25,27], and Bayesian Network [19,36,37]. In addition, ensemble classifiers have become a hotspot in the field of transportation mode detections since ensemble classifiers are a combination set of weak learners, which are superior to a single stronger learner [21,38].…”