Improving Head Pose Estimation with a Combined Loss and Bounding Box Margin Adjustment

Abstract

We address a problem of estimating pose of a person's head from its RGB image. The employment of CNNs for the problem has contributed to significant improvement in accuracy in recent works. However, we show that the following two methods, despite their simplicity, can attain further improvement: (i) proper adjustment of the margin of bounding box of a detected face, and (ii) choice of loss functions. We show that the integration of these two methods achieve the new state-of-the-art on standard benchmark datasets for in-the-wild head pose estimation. The Tensorflow implementation of our work is available at https://github.com/ MingzhenShao/HeadPose

Rotation Axis Focused Attention Network (RAFA-Net) for Estimating Head Pose

“…For example, 300W-LP is a synthetic one, while the BIWI and AFLW2000 consist of real images. The deep learning-based landmark-free approaches such as Hopenet [10], SSR-Net-MD [12], ResNet-BBM [13], FSA-Net [11] and our RAFA-Net perform better than the landmark-based ones (Dlib [1], 3DDFA [31], FAN [2], KEPLER [5] and Two-stage [3]) tested on both the BIWI and AFLW2000 datasets. This is mainly since the landmark-free approaches can better accommodate the domain discrepancies between training and testing datasets.…”

“…The overall performance (MAE) of our RAFA-Net is inferior to the FSA-Caps-Fusion [11] and SSR-Net-MD [12] landmark-free approaches (Table 1). However, it is better than the ResNet-BBM [13] and Hopenet [10]. Moreover, the estimated average error in pitch is better (6.26) than the landmark-free approaches except for the FSA-Caps-Fusion [11] (4.96).…”

“…However, these approaches require manually annotated ground-truth, which is laborious, time-consuming, and often experts cannot accurately assign landmark locations in low-resolution images. Landmark-free approaches: To address the above drawback, recently, there is a significant interest in estimating head poses directly from the image intensities using deep networks [9][10][11][12][13][14]. Such approaches often encounter problems due to illumination variations or poor illumination during night time.…”

“…2) train the model using 70% of videos (16 videos) in the BIWI dataset and evaluate the rest 30% (8 videos). In all three datasets, we use the detected face bounding box provided by Shao et al [13]. The standard evaluation metric of mean absolute error (MAE) is used.…”

“…We propose a novel data augmentation approach (Fig 4) and is inspired by the experiment carried out by Shao et al [13] to measure the accuracy of their We experimentally found that this randomization gives better generalization resulting in improved performance rather than using standard augmentation techniques such as random scaling, width and/or height sifting and cropping. For all our experiments, we have used 0 ≤ γ ≤ 0.5.…”