A Multi-Gesture Interaction System Using a 3-D Iris Disk Model for Gaze Estimation and an Active Appearance Model for 3-D Hand Pointing

Abstract

Abstract-In this paper, we present a vision-based human-computer interaction system, which integrates control components using multiple gestures, including eye gaze, head pose, hand pointing, and mouth motions. To track head, eye, and mouth movements, we present a two-camera system that detects the face from a fixed, wide-angle camera, estimates a rough location for the eye region using an eye detector based on topographic features, and directs another active pan-tilt-zoom camera to focus in on this eye region. We also propose a novel eye gaze estimation approach for point-of-regard (POR) tracking on a viewing screen. To allow for greater head pose freedom, we developed a new calibration approach to find the 3-D eyeball location, eyeball radius, and fovea position. Moreover, in order to get the optical axis, we create a 3-D iris disk by mapping both the iris center and iris contour points to the eyeball sphere. We then rotate the fovea accordingly and compute the final, visual axis gaze direction. This part of the system permits natural, non-intrusive, pose-invariant POR estimation from a distance without resorting to infrared or complex hardware setups. We also propose and integrate a two-camera hand pointing estimation algorithm for hand gesture tracking in 3-D from a distance. The algorithms of gaze pointing and hand finger pointing are evaluated individually, and the feasibility of the entire system is validated through two interactive information visualization applications.Index Terms-Gaze estimation, hand tracking, human-computer interaction (HCI).

A 3D-Eyeball/Skin Decorrelated Active Appearance Model

“…Yamazoe et al (19) also apply a similar head-eye model. It differs from that of Reale (18) in the fact that the 3D model is projected on the image plane and not the contrary. In our method, the iris shape is projected on the eyeball and then re-projected on the 2D plane.…”

Integrating Head Pose to A 3D Multitexture Approach for Gaze Detection

“…They exploit both of the shape and appearance of the eye to find the iris location. Some methods present manually designed models [25,22]. To avoid manual design of the eye model, one can train Active Appearance Models (AAM) [7] on real data.…”

A Regression-Based User Calibration Framework for Real-Time Gaze Estimation

“…We implemented all other investigated calibration methods ourselves. In addition, SDM face tracker 4 is used for the localization of facial features prior to gaze features detection.…”