Generative 3D Hand Tracking with Spatially Constrained Pose Sampling

Abstract

We present a method for 3D hand tracking that exploits spatial constraints in the form of end effector (fingertip) locations. The method follows a generative, hypothesize-andtest approach and uses a hierarchical particle filter to track the hand. In contrast to state of the art methods that consider spatial constraints in a soft manner, the proposed approach enforces constraints during the hand pose hypothesis generation phase by sampling in the Reachable Distance Space (RDS). This sampling produces hypotheses that respect both the hands' dynamics and the end effector locations. The data likelihood term is calculated by measuring the discrepancy between the rendered 3D model and the available observations. Experimental results on challenging, ground truth-annotated sequences containing severe hand occlusions demonstrate that the proposed approach outperforms the state of the art in hand tracking accuracy.

Pushing the Envelope for RGB-Based Dense 3D Hand Pose Estimation via Neural Rendering

“…hand gesture as a primary interface for AR/VR. The problem is challenging due to high dimensionality of hand space, pose and shape variations, self-occlusions, etc [52,43,58,65,10,61,40,5,12,63,47,59,37,67,54,53,9,44,31,49,32,20,27,33,7,64]. Most existing methods have focused on recovering sparse hand poses i.e.…”

“…However, the predictions are based on coarse skeletal representations, and no explicit kinematics and geometric mesh constraints are often considered. On the other hand, establishing a personalized hand model requires a generative approach that optimizes the hand model to fit to 2D images [40,33,36,51,49,48]. Optimization-based methods, besides their complexity, are susceptible to local minima and the personalized hand model calibration contradicts the generalization ability for hand shape variations.…”

A Comprehensive Study on Deep Learning-Based 3D Hand Pose Estimation Methods

“…Moreover, since the encoded image representations include all the necessary information that is needed to predict a hand pose, these methods are not computationally expensive. On the other hand, model-based algorithms [53][54][55][56][57][58][59] establish a prior 3D hand model to match the images into the predefined model. At each frame, an algorithm performs an exploration in order to acquire the pose and shape of the hand model that best matches the features extracted from the input image.…”

Filling the Joints: Completion and Recovery of Incomplete 3D Human Poses

“…A post-processing step is employed by Ciotti et al [34] that refines the estimated hand pose using the occlusion cue as a measure of uncertainty. Roditakis et al [41] estimates hand pose during hand-object interaction by considering spatial constraints induced by the observed hand-object contact points. Deep-learning based methods [42][43][44][45][46][47] for human pose estimation use large datasets to learn the space of natural human poses.…”