Highly Accurate and Fully Automatic 3D Head Pose Estimation and Eye Gaze Estimation Using RGB-D Sensors and 3D Morphable Models

Abstract

This work addresses the problem of automatic head pose estimation and its application in 3D gaze estimation using low quality RGB-D sensors without any subject cooperation or manual intervention. The previous works on 3D head pose estimation using RGB-D sensors require either an offline step for supervised learning or 3D head model construction, which may require manual intervention or subject cooperation for complete head model reconstruction. In this paper, we propose a 3D pose estimator based on low quality depth data, which is not limited by any of the aforementioned steps. Instead, the proposed technique relies on modeling the subject’s face in 3D rather than the complete head, which, in turn, relaxes all of the constraints in the previous works. The proposed method is robust, highly accurate and fully automatic. Moreover, it does not need any offline step. Unlike some of the previous works, the method only uses depth data for pose estimation. The experimental results on the Biwi head pose database confirm the efficiency of our algorithm in handling large pose variations and partial occlusion. We also evaluated the performance of our algorithm on IDIAP database for 3D head pose and eye gaze estimation.

Head Pose Estimation through Keypoints Matching between Reconstructed 3D Face Model and 2D Image

“…Comparisons on the BIWI Dataset. The proposed method is also compared with state-of-the-art methods, including CNN-syn [ 30 ], DNN [ 60 ], regression [ 61 ], Two-Stage [ 62 ], KEPLER [ 25 ], QuatNet [ 63 ], Dlib [ 51 ], FAN [ 48 ], 3DDFA [ 37 ], QT_PYR [ 35 ], 4C_4S_var4 [ 36 ], Haar-Like(LBP) [ 64 ] and HAFA [ 65 ] on the BIWI dataset. We divided these 14 methods into two groups.…”

Comparative Analysis of Kinect-Based and Oculus-Based Gaze Region Estimation Methods in a Driving Simulator

“…The performance of their method was comparable to other approaches although they require significant constraints, invasive hardware, and supervised learning. Ghiass et al [29] proposed a generic and fully automatic method to obtain the facial pose using the Kinect device. They also evaluated their method for 3D gaze estimation.…”

“…In the previously cited works using the Kinect [27][28][29], this device was used for gaze estimation although they have not only used head orientation for it. The use of additional devices and eye detection or tracking makes those approaches more complex and difficult to implement in vehicular environments.…”