Kinect depth stream pre-processing for hand gesture recognition

Abstract

This is the unspecified version of the paper.This version of the publication may differ from the final published version. ABSTRACT Over the recent years there has been growing interest to propose a robust and efficient hand gesture recognition (HGR) system, using real-time depth sensors like Microsoft Kinect. The performance of such HGR systems have been affected by the low resolution, noise and quantization error in the depth stream. In this paper, we propose a method to pre-process Kinect depth stream in order to overcome some of these limitations. The design approach utilizes the hand tracker from OpenNI SDK to perform distance invariant segmentation of hand region depth stream. This is followed by the construction of three different projections of hand in XY, ZX and ZY planes. These projections are then further enhanced using a combination of morphological closing and simple averaging based interpolation. The evaluation results show above 80% similarity with ground truth, and 1.45-5.35% increase in accuracy for gestures with recognition accuracy less than 90%. Permanent repository link

Sujeción de objetos virtuales mediante un gesto manual usando Kinect y software libre

“…Muchos enfoques se han propuesto para buscar reconocer gestos a través de cámaras TOF. Primeramente, se encuentran aquellos que solo usan la cámara de profundidad para realizar el seguimiento y la segmentación de la mano, como propone [5]. La propuesta consiste en un algoritmo que consta de cinco partes: segmentación, extracción de proyección, reducción de error, búsqueda de contorno y aplicación de clasificador basado en redes neuronales.…”

“…El mapeo de las dos imágenes en una sola, proporciona la ventaja usar solo una imagen 2D, lo que mejora el rendimiento. A diferencia de lo que menciona [5], se ha podido comprobar en este trabajo que la representación 2D si aporta la información necesaria para rastrear la posición de la mano.…”

Survey on 3D Hand Gesture Recognition

“…An EWMA (Exponential Weighted Moving Average) noise reduction mechanism was used to suppress the noise effects of the neural network. Meanwhile, a neural network model was constructed to recognize four gesture stages [130]. In both works, the neural network only handled fundamental and limited hand gestures.…”

Hand Gesture Recognition Based on Signals Cross-Correlation

“…One of the covered topics includes hand detection, pose estimation and gesture classification. Hand detection and pose estimation can be accomplished either on depth images [10,19,20,22,28,30,34] or by combination of color and depth information [13,25,33]. The compromise is fast against precise algorithms.…”

“…The most used depth similarity measure between observed and trained images is the inverse of their pixelwise Euclidean distance. The used techniques for hand detection from depth images are simple heuristics [30,34], distance invariant hand segmentation [10,28] or clustering of the depth pixels [10,20] followed by convex hull analysis [20], morphological constraints [10] or a Finger-Earth Mover's distance [30] to measure the dissimilarities between different hand contour/shapes. The critical part here is that depth threshold needs to be determined [34] to indicate the depth level where the hand is located or a variety of authors' assumptions need to be fulfilled, such as the hand to be the front most object or black belt equipped as in [30].…”