The Influence of Speed and Position in Dynamic Gesture Recognition for Human-Robot Interaction

Abstract

Human communication relies on several aspects beyond the speech. One of them is gestures as they express intentions, interests, feelings, or ideas and complement the speech. Social robots need to interpret these messages to allow a more natural Human-Robot Interaction. In this sense, our aim is to study the effect of position and speed features in dynamic gesture recognition. We use 3D information to extract the user’s skeleton and calculate the normalized position for all of its joints, and using the temporal variation of such positions, we calculate their speeds. Our three datasets are composed of 1355 samples from 30 users. We consider 14 common gestures in HRI involving upper body movements. A set of classification techniques is evaluated testing these three datasets to find what features perform better. Results indicate that the union of both speed and position achieves the best results among the three possibilities, 0.999 of F-score. The combination that performs better to detect dynamic gestures in real time is finally integrated in our social robot with a simple HRI application to run a proof of concept test to check how the proposal behaves in a realistic scenario.

Real-Time Visual Recognition of Ramp Hand Signals for UAS Ground Operations

“…A previous study evaluated more than 126 classification models for the recognition of dynamic gestures, concluding that the neural network and the RF classifier were the most accurate for gesture recognition [6]. After preliminary experimentation, we agree with this conclusion.…”

“…Human-Robot Interaction (HRI) is a multidisciplinary field that addresses effective and efficient communication mechanisms between humans and machines, enabling their collaboration in the execution of a given task. Understanding human interaction and implementing human communication abilities into machines is thus a fundamental aspect of HRI [6]. During human communication, the transmission of intentions, interests and feelings are also important.…”

“…Spatial gestures are easy recognizable, as they do not have a temporal dimension, which usually requires a sequence of related data. The influence of speed and position in the dynamic recognition of gestures is reviewed in [6]. Numerous techniques have been developed to recognise and classify gesture´s sequences, including the Hidden Markov Model (HMM) [11], the Longest Common Subsequence Problem (LCS) [12] or the Viterbi algorithm [13].…”

“…However, this alternative has been tested in the second architecture proposed, and explained in Sect. 6.…”

“…According to Tomaszewska et al [2] more than 26% of reported air traffic accidents occur on the ground, with an estimated cost of 11 M$ per year. Furthermore, the number of ground accidents increased by 200% between 2012 and 2017, mostly due to pure human errors during manoeuvres' execution or in combination with other factors, such as high Categories (6), (7). 1 3 workload, insufficient awareness of the situation or adverse weather [2].…”