Facial Expression Recognition from World Wild Web

Abstract

Recognizing facial expression in a wild setting has remained a challenging task in computer vision. The World Wide Web is a good source of facial images which most of them are captured in uncontrolled conditions. In fact, the Internet is a Word Wild Web of facial images with expressions. This paper presents the results of a new study on collecting, annotating, and analyzing wild facial expressions from the web. Three search engines were queried using 1250 emotion related keywords in six different languages and the retrieved images were mapped by two annotators to six basic expressions and neutral. Deep neural networks and noise modeling were used in three different training scenarios to find how accurately facial expressions can be recognized when trained on noisy images collected from the web using query terms (e.g. happy face, laughing man, etc)? The results of our experiments show that deep neural networks can recognize wild facial expressions with an accuracy of 82.12%.

Context-Aware Emotion Recognition Networks

“…According to the data type, the datasets are grouped into the static and dynamic. Even if static databases for facial expression analysis such as Af-fectNet [43] and FER-Wild [26] collect a large amount of facial expression images from the web, they have only facecropped images not including surrounding context. In addition, EMOTIC [14] do not contain human facial images, as exampled in Fig.…”

“…Most of the datasets that focus on detecting occurrence of expressions, such as CK+ [23] and MMI [24], have been taken in lab-controlled environments. Recently, datasets recorded in the wild condition for including naturalistic emotion states [9,25,26] have attracted much attention. AFEW benchmark [9] of the EMOTIW challenge [27] provides video frames extracted from movies and TV shows, while SFEW database [25] has been built as a static subset of the AFEW.…”

“…AFEW benchmark [9] of the EMOTIW challenge [27] provides video frames extracted from movies and TV shows, while SFEW database [25] has been built as a static subset of the AFEW. FER-Wild [26] database contains 24,000 images that are obtained by querying emotion-related terms from search engines. MS-COCO database [28] has been recently annotated with object attributes, including some emotion categories for human, but the attributes are not intended to be exhaustive for emotion recognition, and not all people are annotated with emotion attributes.…”

A Compact Embedding for Facial Expression Similarity

“…Expression datasets: Several facial expression datasets have been created in the past that consist of face images labeled with discrete emotion categories [4,9,10,11,16,17,31,34,40,41,43,54,55], facial action units [4,34,36,37,43], and strengths of valence and arousal [25,27,28,40,44]. While these datasets played a significant role in the advancement of automatic facial expression analysis in terms of emotion recognition, action unit detection and valence-arousal estimation, they are not the best fit for learning a compact expression embedding space that mimics human visual preferences.…”

Recovering Joint and Individual Components in Facial Data

“…Collecting 'in-the-wild' images of same subjects under different expressions is a very tedious task. In order to improve the performance of RJIVE, we augmented the training set with another 1200 images from the WWB database [38] (400 images for each expression). As it can be observed in Figure 3(c), the 'in-the-wild' train set improved the accuracy of RJIVE in both CK+ and ITW datasets.…”