A Framework for Joint Estimation and Guided Annotation of Facial Action Unit Intensity

Abstract

Manual annotation of facial action units (AUs) is highly tedious and time-consuming. Various methods for automatic coding of AUs have been proposed, however, their performance is still far below of that attained by expert human coders. Several attempts have been made to leverage these methods to reduce the burden of manual coding of AU activations (presence/absence). Nevertheless, this has not been exploited in the context of AU intensity coding, which is a far more difficult task. To this end, we propose an expertdriven probabilistic approach for joint modeling and estimation of AU intensities. Specifically, we introduce a Conditional Random Field model for joint estimation of the AU intensity that updates its predictions in an iterative fashion by relying on expert knowledge of human coders. We show in our experiments on two publicly available datasets of AU intensity (DISFA and FERA2015) that the AU coding process can significantly be facilitated by the proposed approach, allowing human coders to faster make decisions about target AU intensity.

An AI Powered Emotion Recognition System

“…In this study, we have explored the development and application of an emotion detection system using a convolutional neural network (CNN) approach. [14] Our investigation yielded promising results, demonstrating the efficacy of CNN architectures in accurately classifying seven basic emotions: anger, disgust, fear, happiness, sadness, surprise, and neutral. Through rigorous experimentation and validation, we have showcased the potential of deep learning techniques in capturing complex facial expressions and discerning subtle emotional cues.…”

Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units

“…To model relationships among multiple AUs, Sandbach et al [40] proposed a tree-structured Markov Random Field model to capture relationships among intensities. Walecki et al introduced copula functions to jointly estimate the intensities of multiple AUs in [49] and [48]. Kaltwang et al [13] proposed a latent tree model and learned tree structure from input features and labels.…”