Dynamic hand gesture recognition using RGB-D motion history and kernel descriptor

Abstract

Gesture recognition has important applications in sign language and human -machine interfaces. In recent years, recognizing dynamic hand gesture using multi-modal data has become an emerging research topic. The problem is challenging due to the complex movements of hands and the limitations of data acquisition. In this work, we present a new approach for recognizing hand gesture using motion history images (MHI) [1] and a kernel descriptor (KDES) [2]. We propose to use an improved version of MHI for modeling movements of hand gesture, where MHI is computed on both RGB and depth data. We propose some improvements in patch-level feature extraction for KDES, which is then applied to MHI to represent gesture features. Then SVM classifier is trained for recognizing gestures. Experiments have been conducted on challenging hand gesture data set of CHALEARN contest [3]. An extensive investigation has been done to analyze the performance of both improved MHI and KDES on multi-modal data. Experimental results show the state-of-the-art of our approach in comparison to the results of the contest.