POSEidon: Face-from-Depth for Driver Pose Estimation

Abstract

Fast and accurate upper-body and head pose estimation is a key task for automatic monitoring of driver attention, a challenging context characterized by severe illumination changes, occlusions and extreme poses. In this work, we present a new deep learning framework for head localization and pose estimation on depth images. The core of the proposal is a regressive neural network, called POSEidon, which is composed of three independent convolutional nets followed by a fusion layer, specially conceived for understanding the pose by depth. In addition, to recover the intrinsic value of face appearance for understanding head position and orientation, we propose a new Facefrom-Depth model for learning image faces from depth. Results in face reconstruction are qualitatively impressive. We test the proposed framework on two public datasets, namely Biwi Kinect Head Pose and ICT-3DHP, and on Pandora, a new challenging dataset mainly inspired by the automotive setup. Results show that our method overcomes all recent state-of-art works, running in real time at more than 30 frames per second.

Face-from-Depth for Head Pose Estimation on Depth Images

“…However, no previous method that uses depth maps as the only input exploits CNNs in an effective way. In this work we propose a method based on [10] which uses depth maps to produce accurate head pose predictions by leveraging CNNs.…”

“…In the automotive field, vision-based systems are required to cooperate or even replace other traditional sensors, due to the increasing presence of cameras inside new car's cockpits and to the ease of capturing images and videos in a completely non-invasive manner. In the past, encouraging results for driver head pose estimation have been achieved using RGB images [1], [2], [3], [4], [5] as well as different camera types, such as infrared [6], thermal [7], or depth [8], [9], [10]. Among them, the last ones are very promising, since they allow robustness when facing strong illumination variations.…”

Domain Translation with Conditional GANs: from Depth to RGB Face-to-Face

“…Experiments are conducted exploiting two publicly available datasets: Pandora [6] and MotorMark [13]. Pandora contains more than 250k frames, splitted into 110 annotated sequences of 22 different actors (10 males and 12 females), while MotorMark is composed of more than 30k frames of 35 different subjects, guaranteeing a great variety of face appearances.…”

Learning to Generate Facial Depth Maps

“…The Pandora dataset was introduced in [3] for the head pose estimation task in depth images. It consists of more than Table 1.…”