A Robust Hand Gesture Recognition Using Combined Moment Invariants in Hand Shape

Abstract

In this paper, we suggest a method that recognizes hand gesture based on moment features in hand shape. First of all, hand regions are segmented from input streams based on skin color detection. Hand detection can be achieved more easily if smart devices such as Kinect are used but we used web camera as an input device. Because hand and also face can be segmented from a frame, we try to remove face from the segmented result. From segmented hand regions palm region is extracted by removing wrist and then moment invariants are calculated from the palm region. Finally we use artificial neural network to classify the classes of the hand gestures. We perform recognition test for input patters with trained DB of 7 classes that contains hand gesture of rock-paper-scissors game and 3 different kinds of hand shape concerned with robot control.

Shape Recognition Using Combined Moment Invariants and PCA

“…We extract pixels with colors within predefined range of bins in 2D histogram of hue (H) and saturation (S) in HSV color space to select hand candidates [2]. Sometimes faces can be one of hand candidates because faces has similar color distribution with hands.…”

“…Smart electric whiteboards for remote control and drawing at long distance apart from location of whiteboards need hand and gesture detection [1]. These techniques are useful in robot control and entertainments [2].…”

“…Moments are used intensively for many computer vision applications. They are used for static hand gesture recognition [2][3][4] or for dynamic gesture [5]. They are also used for other shape recognition such as Chinses characters [6,7], iris [8], faces [9] and traffic signs [10].…”

“…In this paper, we compare the recognition performance between Hu and Zernike moments. There are some studies to recognize hand gestures by using Hu, Zernike, Tchebichef or Krawtchouk [2][3][4]. We combine two moments of Hu and Zernike to try improve the recognition accuracy.…”

“…By the way, a hand candidate can be consisted of not only hand but also wrist or forearm connected with the hand. So we localize hands from hand candidates with advanced method against our preliminary study [2]. We show intensive evaluation results on recognition about two dataset.…”