Efficient model-based 3D tracking of hand articulations using Kinect

Abstract

We present a novel solution to the problem of recovering and tracking the 3D position, orientation and full articulation of a human hand from markerless visual observations obtained by a Kinect sensor. We treat this as an optimization problem, seeking for the hand model parameters that minimize the discrepancy between the appearance and 3D structure of hypothesized instances of a hand model and actual hand observations. This optimization problem is effectively solved using a variant of Particle Swarm Optimization (PSO). The proposed method does not require special markers and/or a complex image acquisition setup. Being model based, it provides continuous solutions to the problem of tracking hand articulations. Extensive experiments with a prototype GPU-based implementation of the proposed method demonstrate that accurate and robust 3D tracking of hand articulations can be achieved in near real-time (15Hz).

Gravity optimised particle filter for hand tracking

“…The largest skin coloured blob is considered to be the hand [12]. A contour is traced on I bp and DouglasPeucker polygon approximation method [25,24] is used to reduce the number of redundant contour points cp.…”

“…Furthermore, the computational requirements is high. Recent work in [12] addresses this poblem using GPU-based software implementation and off-the-shelf Kinect sensor which demonstrates robust 3D articulated hand tracking in near real-time (15Hz) over a long sequence.…”

“…The modelbased approach [12,13] is based on parametric models of the shape and kinematic structure of the 3-dimensional (3D) hand. Using the motion history and dynamics of the hand, this approach predicts the model hypotheses when a new observation is made by measuring the dissimilarity between the expected model hypotheses with the actual observation.…”

Online handwriting recognition using depth sensors

“…Bunke et al [8] and Fink et al [9] used multiple cameras, whereas [4,5,10,11] used depth sensors for gesture and handwriting recognition. Lee and Lee [12] proposed the use of a skin color model to track the hand and fingertips whereas Oikonomidis et al [13] models the hand using a multi-camera setup. Liwicki and Everingham [5] have worked on recognizing words from video, where words are finger spelled using the British Sign Language(BSL).…”

Dimensionality Reduction for Efficient Single Frame Hand Pose Estimation

“…Body pose can be calculated at several scales and granularities including full body [14,18], head [13] and hand pose [3,4,10,12,19]. Recovering the articulation of a human hand can be proven very useful in a number of application domains including but not limited to advanced HCI/HRI, games, AR applications, sign language understanding, etc.…”