3D fingertip and palm tracking in depth image sequences

Abstract

We present a vision-based approach for robust 3D fingertip and palm tracking on depth images using a single Kinect sensor. First the hand is segmented in the depth images by applying depth and morphological constraints. The palm is located by performing distance transform to the hand contour and tracked with a Kalman filter. The fingertips are detected by combining three depth-based features and tracked with a particle filter over successive frames. Quantitative results on synthetic depth sequences show the proposed scheme can track the fingertips quite accurately. Besides, its capabilities are further demonstrated through a real-life human-computer interaction application.

Guided optimisation through classification and regression for hand pose estimation

“…Krejov [32] and Liang [33] identify the fingertips by considering them geodesic maxima from the centre of the hand. Such approaches are very fast to compute, and in the case of [32], four hands can be tracked simultaneously.…”

Hand landmarks detection and localization in color images

“…Importantly, retrieving more information on hand landmarks location may improve the accuracy of the state-of-the-art techniques, especially if it were possible to find the position of the landmarks located inside the hand silhouettes. It has been achieved only for the depth maps [29][30][31]52], acquired using the Kinect sensor or timeof-flight (ToF) cameras, and this has substantially improved the accuracy and reliability of gesture-controlled interfaces.…”

“…Seldom has it been attempted to detect other landmarks associated with particular knuckles [45,51]. Existing approaches may be categorized into those that exploit (i) template matching [17,25,39,45], (ii) distance transform [6,30] and (iii) contour analysis [9,14,42,54].…”

“…This approach makes it possible to estimate the finger direction with high precision and allows for detecting the landmarks of extended fingers, but it is less effective for folded digits. A similar approach was applied by Liang et al [30] to detect the fingertips from depth images acquired using the Kinect sensor. It is assumed that the hand is formed by the largest skin-color blob, and the palm center is determined from the circle inscribed into the blob.…”

3D Visual Sensing of the Human Hand for the Remote Operation of a Robotic Hand

“…The critical aspect of this was to determine a threshold, indicating the depth of the level at which the hand was located. Liang et al [14] detected hands in depth images using a clustering algorithm followed by morphological constraints. Afterwards, a distance transform to the segmented hand contour was performed to estimate the palm and its centre.…”