Constraint-Based Hand Animation

Immersive virtual environments with life-like interaction capabilities have very demanding requirements including high-precision motion capture and high-processing speed. These issues raise many challenges for computer vision-based motion estimation algorithms. In this study, we consider the problem of hand tracking using multiple cameras and estimating its 3D global pose (i.e., position and orientation of the palm). Our interest is in developing an accurate and robust algorithm to be employed in an immersive virtual training environment, called "Virtual GloveboX" (VGX) (Twombly et al. in J Syst Cybern Inf 2:30-34, 2005), which is currently under development at NASA Ames. In this context, we present a marker-based, hand tracking and 3D global pose estimation algorithm that operates in a controlled, multi-camera, environment built to track the user's hand inside VGX. The key idea of the proposed algorithm is tracking the 3D position and orientation of an elliptical marker placed on the dorsal part of the hand using modelbased tracking approaches and active camera selection. It should be noted that, the use of markers is well justified in the context of our application since VGX naturally allows for the use of gloves without disrupting the fidelity of the interaction. Our experimental results and comparisons illustrate that the proposed approach is more accurate and robust than related approaches. A byproduct of our multi-camera ellipse tracking algorithm is that, with only minor modifications, the same algorithm can be used to automatically re-calibrate (i.e., fine-tune) the extrinsic parameters of a multi-camera system leading to more accurate pose estimates.

show abstract

“…Therefore, there have been many attempts mostly using gloves painted with point markers [4][5][6][7][8][9][10].…”

Section: Previous Workmentioning

confidence: 99%

“…Lien et al [7,10] derived closed form solutions using regression techniques to map fingertip positions to joint angles. In a much older study, Lee et al [8] presented a model fitting algorithm that gradually updates the joint angles based on the finger motion constraints to reach the extracted fingertip locations. Holden el al.…”

Section: Previous Workmentioning

confidence: 99%

Global hand pose estimation by multiple camera ellipse tracking

Usabiaga

Erol

Bebis

et al. 2008

Machine Vision and Applications

View full text Add to dashboard Cite

show abstract

“…Based on visual investigations, (Gavrila, 1999), the human hand has been modeled as a multi-DoF rigid body system, (Huang, 1990), and deformable body system . To consider the rules, constraint-based modeling, (Lee and Kunii, 1993), and knowledgeintensive animation of hand grasping (Rijpkema and Girard, 1991) have been proposed. Tracking of the positions and orientations of the hand can be by visionbased and non-vision-based methods such as magnetic, acoustic, and inertial tracking.…”

Section: Research In Manipulator Indicative and Descriptive Functionmentioning

confidence: 99%

Using Hand Motions in Conceptual Shape Design: Theories, Methods and Tools

Varga

2008

Product Engineering

View full text Add to dashboard Cite

“…Very simple models for the upper-body use just 10 DoF (3 DoF for each shoulder and 2 DoF for each elbow), while more complex skeletons may use up to 50 DoF [1,4]. A common skeleton model for the human hand is the one proposed in [32], made by 27 bones and 19 joints (see Fig. 8.2).…”

mentioning

confidence: 99%

Human Pose Estimation and Tracking

Zanuttigh

Marin

Mutto³

et al. 2016

Time-of-Flight and Structured Light Depth Cameras

View full text Add to dashboard Cite

The estimation of the movements and posture of human beings is one of the key applications of consumer depth cameras. It motivated the development of the Kinect TM v1 and v2, and favored the diffusion of ToF and structured light technologies from the industrial and research fields to the mass market. The appeal of human pose estimation and tracking is due to its vast range of applications solving daily life tasks. Console games using the body or the hands as controller were the first commercial application of consumer depth cameras, and the skeletal tracking approach introduced with Kinect TM v1 represents the first reliable and efficient solution to the pose estimation problem in a home environment. Humancomputer interaction is another intriguing field, as the various hand configurations and body movements are often exploited to convey non-verbal information, either by explicitly associating gestures to specific meanings or more implicitly by augmenting speech information. Besides people interaction, human pose can also have a fundamental role in many situations requiring the manipulation of an object or the possibility of controlling a machine by performing intuitive (natural) movements, e.g., in the robotics field. Historically, computer animation has been one of the first and more active areas successfully exploiting human pose data derived from motion capture. Complex movements performed by a human actor can be tracked and recorded in order to be used either in real-time or in a second time, to drive the movements of some computer-generated character or avatar (motion retargeting). Finally, many other applications exploit information from human pose estimation and tracking, e.g., video surveillance and control, posture and movement analysis in medical applications and data compression through the use of representations more compact than full 3D point clouds.Various solutions have been proposed for human pose estimation and tracking task ( Fig. 8.1). Marker-based systems are able to acquire reliable information about body or hand posture but they are expensive and invasive, therefore their usage is confined to highly controlled industrial or medical environments. Colored gloves and special suits equipped with reflective or LED lights markers require delicate Even though the geometric 3D information embedded in depth data can solve some of the problems of systems based on a single color camera, such approach requires to properly solve a number of issues. For example, the depth data provided by many depth cameras are affected by a considerable amount of noise and artifacts. Moreover, single-view approaches based on 3D geometry often present a large number of self-occlusions, resulting in missing data. Deploying multiple depth cameras (whenever the mutual interference is negligible) reduces missing data at the expenses of more complex calibration procedures. Self-occlusions probably represent one of the most difficult problems within single view pose recovery, and as well known even multi-view acquisition setups...

show abstract

Constraint-Based Hand Animation

Cited by 48 publications

References 8 publications

Global hand pose estimation by multiple camera ellipse tracking

Global hand pose estimation by multiple camera ellipse tracking

Using Hand Motions in Conceptual Shape Design: Theories, Methods and Tools

Human Pose Estimation and Tracking

Contact Info

Product

Resources

About