IdenNet: Identity-Aware Facial Action Unit Detection

Tu, Cheng-Hao; Yang, Chih‐Yuan; Hsu, Jane Yung-jen

doi:10.1109/fg.2019.8756631

Cited by 17 publications

(14 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Fan et al [8] and Niu et al [22] proposed methods that use facial landmarks to take individuality of facial shapes into account. Tu et al [25] studied a method to discriminate appearance changes caused by facial expression and individuality through using an additional identityannotated dataset. Other existing methods considering temporal features have also been studied.…”

Section: Related Work About Deep Neural Networkmentioning

confidence: 99%

Action Units Recognition Using Improved Pairwise Deep Architecture

Saito,

Mi,

Uchida

et al. 2021

Preprint

View full text Add to dashboard Cite

Facial Action Units (AUs) represent a set of facial muscular activities and various combinations of AUs can represent a wide range of emotions. AU recognition is often used in many applications, including marketing, healthcare, education, and so forth. Although a lot of studies have developed various methods to improve recognition accuracy, it still remains a major challenge for AU recognition. In the Affective Behavior Analysis in-the-wild (ABAW) 2020 competition, we proposed a new automatic Action Units (AUs) recognition method using a pairwise deep architecture to derive the Pseudo-Intensities of each AU and then convert them into predicted intensities. This year, we introduced a new technique to last year's framework to further reduce AU recognition errors due to temporary face occlusion such as hands on face or large face orientation. We obtained a score of 0.65 in the validation data set for this year's competition.

show abstract

Section: Related Work About Deep Neural Networkmentioning

confidence: 99%

Action Units Recognition Using Improved Pairwise Deep Architecture

Saito,

Mi,

Uchida

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Subsequently, appearance features learned by CNNs have been immensely used in facial expression analysis in the last few years [22], [19]. Some works used relatively shallow CNNs (with 3-5 convolutional layers) [8], [13], [15], [3], [34], while others used deeper CNNs, that have been trained on other classification tasks like object or face recognition [39], [2], [10], [35]. The computational and memory cost of the AU detection pipeline increases notably as the CNN complexity increases.…”

Section: Cnn Architecturesmentioning

confidence: 99%

“…Some works in AU detection used the raw face images as input to the CNNs [14], [39], [3], [6], [2], [34], while others centered (i.e. subtract the mean) or normalized the images (i.e.…”

Section: A Input Centering/normalizationmentioning

confidence: 99%

“…Different CNN architectures have been used across the literature for AU detection. Some works used shallow CNNs for feature extraction [8], [15], [34], while others trained deeper CNNs with relatively more layers, filters, and parameters [39], [10]. Deeper CNNs increase the computational and memory cost of the whole AU detection pipeline, however, it is not clear how such deep CNNs perform in comparison to other shallow CNNs, as well as how those deep CNNs compare to each other.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Which CNNs and Training Settings to Choose for Action Unit Detection? A Study Based on a Large-Scale Dataset

Bishay¹,

Ghoneim²,

Ashraf³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper we explore the influence of some frequently used Convolutional Neural Networks (CNNs), training settings, and training set structures, on Action Unit (AU) detection. Specifically, we first compare 10 different shallow and deep CNNs in AU detection. Second, we investigate how the different training settings (i.e. centering/normalizing the inputs, using different augmentation severities, and balancing the data) impact the performance in AU detection. Third, we explore the effect of increasing the number of labelled subjects and frames in the training set on the AU detection performance. These comparisons provide the research community with useful tips about the choice of different CNNs and training settings in AU detection. In our analysis, we use a large-scale naturalistic dataset, consisting of ∼55K videos captured in the wild. To the best of our knowledge, there is no work that had investigated the impact of such settings on a large-scale AU dataset.

show abstract

“…Peng et al [14] propose a weakly supervised approach with adversarial training using domain knowledge such as dependencies between Action Units. Tu et al [15] focus on identity-dependent image features, which they extract by a face clustering network. Due to the different approaches, it is difficult to compare the results, but it shows nevertheless the competitive generalization ability of our network.…”

Section: Resnetmentioning

confidence: 99%

Verifying Deep Learning-based Decisions for Facial Expression Recognition

Rieger,

Kollmann,

Finzel

et al. 2020

Preprint

View full text Add to dashboard Cite

Neural networks with high performance can still be biased towards non-relevant features. However, reliability and robustness is especially important for high-risk fields such as clinical pain treatment. We therefore propose a verification pipeline, which consists of three steps. First, we classify facial expressions with a neural network. Next, we apply layer-wise relevance propagation to create pixel-based explanations. Finally, we quantify these visual explanations based on a bounding-box method with respect to facial regions. Although our results show that the neural network achieves state-of-the-art results, the evaluation of the visual explanations reveals that relevant facial regions may not be considered.

show abstract

IdenNet: Identity-Aware Facial Action Unit Detection

Cited by 17 publications

References 32 publications

Action Units Recognition Using Improved Pairwise Deep Architecture

Action Units Recognition Using Improved Pairwise Deep Architecture

Which CNNs and Training Settings to Choose for Action Unit Detection? A Study Based on a Large-Scale Dataset

Verifying Deep Learning-based Decisions for Facial Expression Recognition

Contact Info

Product

Resources

About