Face Alignment Using K-Cluster Regression Forests With Weighted Splitting

Abstract

Abstract-In this work we present a face alignment pipeline based on two novel methods: weighted splitting for K-cluster Regression Forests and 3D Affine Pose Regression for face shape initialization. Our face alignment method is based on the Local Binary Feature framework, where instead of standard regression forests and pixel difference features used in the original method, we use our K-cluster Regression Forests with Weighted Splitting (KRFWS) and Pyramid HOG features. We also use KRFWS to perform Affine Pose Regression (APR) and 3D-Affine Pose Regression (3D-APR), which intend to improve the face shape initialization. APR applies a rigid 2D transform to the initial face shape that compensates for inaccuracy in the initial face location, size and in-plane rotation. 3D-APR estimates the parameters of a 3D transform that additionally compensates for out-of-plane rotation. The resulting pipeline, consisting of APR and 3D-APR followed by face alignment, shows an improvement of 20% over standard LBF on the challenging IBUG dataset, and state-of-theart accuracy on the entire 300-W dataset.

Deep Alignment Network: A Convolutional Neural Network for Robust Face Alignment

“…The annotation for the images in the 300W public test set were originally published for the 300W competition as part of its training set. We use them for testing as it became a common practice to do so in the recent years [21,35,31,18,16].…”

“…Face alignment has a long history, starting with the early Active Appearance Models [5,20], moving to Constrained Local Models [7,1] and recently shifting to methods based on Cascaded Shape Regression (CSR) [32,4,21,18,16,30,13] and deep learning [34,10,29,31,3].…”

A CPU Real-Time Face Alignment for Mobile Platform

“…The development of face alignment starts from the earliest Active Appearance Models [38], [39] and Constrained Local Models [50], [51], moving to Cascaded Shape Regression (CSR) [4], [3], [6], [7], [40], [41] and deep learning methods [42], [43], [20], [44], [45]. The traditional face alignment cascade regression methods have some research results which show that the good face shape initialization can improve the regression results [4], [8], [20], [36], [37], so we use the difference between the ground truth and mean shape as the learning label of the first stage.…”

HoloFace: Augmenting Human-to-Human Interactions on HoloLens

“…Face alignment is the most crucial part of our framework that all the other elements are based on. For that reason in HoloFace we implement two state-of-the-art face alignment methods: a method based on regression trees which is capable of running locally on the device [14] and a more powerful method based on deep neural networks which is intended to run on a remote desktop machine [15].…”

“…The first method, which we will refer to as KRFWS, is based on the work of Kowalski et al in [14]. The authors of [14] propose a face alignment pipeline that uses novel K-Cluster Regression Forests with Weighted Splitting.…”