Exploiting Depth and Intensity Information for Head Pose Estimation with Random Forests and Tensor Models

Kaymak, Sertan; Patras, Ioannis

doi:10.1007/978-3-642-37484-5_14

Cited by 7 publications

(4 citation statements)

References 18 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HOG features [43] were extracted from RGB and depth images in [44], [45], then a Multi Layer Perceptron and a linear SVM [46] were used for feature classification, respectively. In [47] Random Forests and tensor regression algorithms are exploited while [48] used a cascade of tree classifiers to tackle extreme head pose estimation task. Recently, in [49] a multimodal CNN was proposed to estimate gaze direction: a regression approach was only approximated through a classifier with a granularity of 1 • and with 360 classes.…”

Section: Related Workmentioning

confidence: 99%

Face-from-Depth for Head Pose Estimation on Depth Images

Borghi

Fabbri

Vezzani

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Depth cameras allow to set up reliable solutions for people monitoring and behavior understanding, especially when unstable or poor illumination conditions make unusable common RGB sensors. Therefore, we propose a complete framework for the estimation of the head and shoulder pose based on depth images only. A head detection and localization module is also included, in order to develop a complete end-to-end system. The core element of the framework is a Convolutional Neural Network, called POSEidon + , that receives as input three types of images and provides the 3D angles of the pose as output. Moreover, a Face-from-Depth component based on a Deterministic Conditional GAN model is able to hallucinate a face from the corresponding depth image. We empirically demonstrate that this positively impacts the system performances. We test the proposed framework on two public datasets, namely Biwi Kinect Head Pose and ICT-3DHP, and on Pandora, a new challenging dataset mainly inspired by the automotive setup. Experimental results show that our method overcomes several recent state-of-art works based on both intensity and depth input data, running in real-time at more than 30 frames per second.

show abstract

Section: Related Workmentioning

confidence: 99%

Face-from-Depth for Head Pose Estimation on Depth Images

Borghi

Fabbri

Vezzani

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…However, this approach suffers from drift problems, where the final model is not well aligned with target's 3D position. Some other proposals propose to combine depth and color cues using random forest [32]. Here, tensor-based regressors allow to model large variations of head orientation.…”

Section: Rgb-d-based Approachesmentioning

confidence: 99%

Robust head pose estimation based on key frames for human-machine interaction

Madrigal

Lerasle

2020

J Image Video Proc.

View full text Add to dashboard Cite

Humans can interact with several kinds of machine (motor vehicle, robots, among others) in different ways. One way is through his/her head pose. In this work, we propose a head pose estimation framework that combines 2D and 3D cues using the concept of key frames (KFs). KFs are a set of frames learned automatically offline that consist the following: 2D features, encoded through Speeded Up Robust Feature (SURF) descriptors; 3D information, captured by Fast Point Feature Histogram (FPFH) descriptors; and target's head orientation (pose) in real-world coordinates, which is represented through a 3D facial model. Then, the KF information is re-enforced through a global optimization process that minimizes error in a way similar to bundle adjustment. The KF allows to formulate, in an online process, a hypothesis of the head pose in new images that is then refined through an optimization process, performed by the iterative closest point (ICP) algorithm. This KF-based framework can handle partial occlusions and extreme rotations even with noisy depth data, improving the accuracy of pose estimation and detection rate. We evaluate the proposal using two public benchmarks in the state of the art: (1) BIWI Kinect Head Pose Database and (2) ICT 3D HeadPose Database. In addition, we evaluate this framework with a small but challenging dataset of our own authorship where the targets perform more complex behaviors than those in the aforementioned public datasets. We show how our approach outperforms relevant state-of-the-art proposals on all these datasets.

show abstract

“…Other works propose to combine the color and depth cues [16]. The work of Kaymak and Patras [5] is similar to the approach of Fanelli, it uses random forests with tensor regression methods to model large variations of head pose. Smolyanskiy et al [15] include a new constraint on the AAM model fitting that considers the depth.…”

Section: Rgbd-based Approachesmentioning

confidence: 99%

3D Head Pose Estimation Enhanced Through SURF-Based Key-Frames

Madrigal

Lerasle

Monin

2018

2018 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

This work presents a method that incorporates 2D and 3D cues for the estimation of head pose. We propose the use of the concept of Key-Frames (KF), a set of frames where the position and orientation of the head is automatically calculated off-line, to improve the precision of pose estimation and detection rate. Each KF consists of: 2D information, encoded by SURF descriptors; 3D information from a depth image (both acquired by an RGB-D sensor); and a generic 3D model that corresponds to the head localization and orientation in the real world. Our algorithm compares a new frame against all KFs and selects the most relevant one. The 3D transformation between both, selected KF and current frame, can be estimated using the depth image and the Iterative Closest Point algorithm in an online framework. Compared to reference approaches, our system can handle partial occlusions and extreme rotations even with noisy depth data. We evaluate the proposal using two challenging datasets: (1) an dataset acquired by us where the groundtruth information is given by a commercial Motion Capture system and (2) the public benchmark Biwi Kinect Head Pose Database.

show abstract

Exploiting Depth and Intensity Information for Head Pose Estimation with Random Forests and Tensor Models

Cited by 7 publications

References 18 publications

Face-from-Depth for Head Pose Estimation on Depth Images

Face-from-Depth for Head Pose Estimation on Depth Images

Robust head pose estimation based on key frames for human-machine interaction

3D Head Pose Estimation Enhanced Through SURF-Based Key-Frames

Contact Info

Product

Resources

About