Using geometric properties of topographic manifold to detect and track eyes for human-computer interaction

Abstract

Automatic eye detection and tracking is an important component for advanced human-computer interface design. Accurate eye localization can help develop a successful system for face recognition and emotion identification. In this article, we propose a novel approach to detect and track eyes using geometric surface features on topographic manifold of eye images. First, in the joint spatial-intensity domain, a facial image is treated as a 3D terrain surface or image topographic manifold. In particular, eye regions exhibit certain intrinsic geometric traits on this topographic manifold, namely, the pit -labeled center and hillside -like surround regions. Applying a terrain classification procedure on the topographic manifold of facial images, each location of the manifold can be labeled to generate a terrain map. We use the distribution of terrain labels to represent the eye terrain pattern. The Bhattacharyya affinity is employed to measure the distribution similarity between two topographic manifolds. Based on the Bhattacharyya kernel, a support vector machine is applied for selecting proper eye pairs from the pit-labeled candidates. Second, given detected eyes on the first frame of a video sequence, a mutual-information-based fitting function is defined to describe the similarity between two terrain surfaces of neighboring frames. By optimizing the fitting function, eye locations are updated for subsequent frames. The distinction of the proposed approach lies in that both eye detection and eye tracking are performed on the derived topographic manifold, rather than on an original-intensity image domain. The robustness of the approach is demonstrated under various imaging conditions and with different facial appearances, using both static images and video sequences without background constraints.

A literature survey on robust and efficient eye localization in real-life scenarios

“…Alternatively, Wang et al [99] developed a facial landscape navigation technique for eye localization, in which the interested intensity pattern (a pit at the eye center surrounded by hillside) is searched in a 3D terrain surface manifold of the face. In a similar method, gradient patterns [53] instead of intensity patterns are calculated around the eye region and served as the template for eye matching.…”

A Multi-Gesture Interaction System Using a 3-D Iris Disk Model for Gaze Estimation and an Active Appearance Model for 3-D Hand Pointing

“…Alternatively, eye gaze may be determined from the pupil or iris contours [17] using ellipse fitting approaches [18], [19]. With a sufficiently large image of the eye, the iris contour and reflection of the screen off the cornea can be used to determine gaze [20].…”

“…Once the face is detected, we also perform eye detection using the so-called topographic features of eyes presented in our previous work [19]. The basic idea is to create a terrain map from the gray-scale image, effectively treating it like a continuous 3-D surface, and extract "pit" locations as pupil candidates.…”

“…The approximate center of the two detected eye candidates is used as the focus of the pan-tilt-zoom camera. Note that we do not use the mutual information tracking approach presented in [19] and instead detect the eyes once every time the face is detected in the wide-angle view.…”

“…The intersection of and is equivalent to the intersection of with a plane with point and normal . So (19) We work out : (20) Substituting this for in (19) (21)…”