Copula Ordinal Regression for Joint Estimation of Facial Action Unit Intensity

Walecki, Robert; Rudovic, Ognjen; Pavlović, Vladimir; Pantić, Maja

doi:10.1109/cvpr.2016.530

Cited by 39 publications

(24 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A facial landmark usually has specific semantic meaning, e.g. nose tip or eye centre, which provides rich geometric information for other face analysis tasks such as face recognition [57,42,39,69], emotion estimation [71,16,59,37] and 3D face reconstruction [15,33,28,27,50,35,19].…”

Section: Introductionmentioning

confidence: 99%

Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks

Feng

Kittler

Awais

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

359

212

View full text Add to dashboard Cite

We present a new loss function, namely Wing loss, for robust facial landmark localisation with Convolutional Neural Networks (CNNs). We first compare and analyse different loss functions including L2, L1 and smooth L1. The analysis of these loss functions suggests that, for the training of a CNN-based localisation model, more attention should be paid to small and medium range errors. To this end, we design a piece-wise loss function. The new loss amplifies the impact of errors from the interval (-w, w) by switching from L1 loss to a modified logarithm function.To address the problem of under-representation of samples with large out-of-plane head rotations in the training set, we propose a simple but effective boosting strategy, referred to as pose-based data balancing. In particular, we deal with the data imbalance problem by duplicating the minority training samples and perturbing them by injecting random image rotation, bounding box translation and other data augmentation approaches. Last, the proposed approach is extended to create a two-stage framework for robust facial landmark localisation. The experimental results obtained on AFLW and 300W demonstrate the merits of the Wing loss function, and prove the superiority of the proposed method over the state-of-the-art approaches.

show abstract

Section: Introductionmentioning

confidence: 99%

Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks

Feng

Kittler

Awais

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

359

212

View full text Add to dashboard Cite

show abstract

“…We do so by means of copula functions [4], known for their ability to capture highly non-linear dependencies through a simple parametrization. The notion of the copula functions has previously been explored for modeling of structured output [42] but not in the context of structured deep learning.…”

Section: Introductionmentioning

confidence: 99%

Deep Structured Learning for Facial Action Unit Intensity Estimation

Walecki

Rudovic

Pavlović

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

112

View full text Add to dashboard Cite

We consider the task of automated estimation of facial expression intensity. This involves estimation of multiple output variables (facial action units -AUs) that are structurally dependent. Their structure arises from statistically induced co-occurrence patterns of AU intensity levels. Modeling this structure is critical for improving the estimation performance; however, this performance is bounded by the quality of the input features extracted from face images. The goal of this paper is to model these structures and estimate complex feature representations simultaneously by combining conditional random field (CRF) encoded AU dependencies with deep learning. To this end, we propose a novel Copula CNN deep learning approach for modeling multivariate ordinal variables. Our model accounts for ordinal structure in output variables and their non-linear dependencies via copula functions modeled as cliques of a CRF. These are jointly optimized with deep CNN feature encoding layers using a newly introduced balanced batch iterative training algorithm. We demonstrate the effectiveness of our approach on the task of AU intensity estimation on two benchmark datasets. We show that joint learning of the deep features and the target output structure results in significant performance gains compared to existing deep structured models for analysis of facial expressions.

show abstract

“…We can draw an analogy between modeling facial AUs and modeling news media, where each medium expresses a particular bias (political ideology) and can also be associated with a particular level of factuality. Therefore, bias and factuality can be analogous to the facial AUs in (Walecki et al, 2016), and represent two aspects of news reporting, each being modeled on a multi-point ordinal scale. In particular, we model bias on a 7-point scale (extreme-left, left, center-left, center, centerright, right, and extreme-right), and factuality on a 3-point scale (low, mixed, and high).…”

Section: Methodsmentioning

confidence: 99%

Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media

Baly¹,

Karadzhov²,

Saleh³

et al. 2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

In the context of fake news, bias, and propaganda, we study two important but relatively under-explored problems: (i) trustworthiness estimation (on a 3-point scale) and (ii) political ideology detection (left/right bias on a 7-point scale) of entire news outlets, as opposed to evaluating individual articles. In particular, we propose a multi-task ordinal regression framework that models the two problems jointly. This is motivated by the observation that hyper-partisanship is often linked to low trustworthiness, e.g., appealing to emotions rather than sticking to the facts, while center media tend to be generally more impartial and trustworthy. We further use several auxiliary tasks, modeling centrality, hyperpartisanship, as well as left-vs.-right bias on a coarse-grained scale. The evaluation results show sizable performance gains by the joint models over models that target the problems in isolation.

show abstract

Copula Ordinal Regression for Joint Estimation of Facial Action Unit Intensity

Cited by 39 publications

References 26 publications

Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks

Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks

Deep Structured Learning for Facial Action Unit Intensity Estimation

Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media

Contact Info

Product

Resources

About