AFAR: A Deep Learning Based Tool for Automated Facial Affect Recognition

Abstract

I. MOTIVATIONAutomated facial affect recognition is crucial to multiple domains (e.g., health, education, entertainment). Commercial tools are available but costly and of unknown validity. Opensource ones [1] lack user-friendly GUI for use by nonprogrammers. For both types, evidence of domain transfer and options for retraining for use in new domains typically are lacking.Deep approaches have two key advantages. They typically outperform shallow ones for facial affect recognition [2], [3], [4]. And pre-trained models provided by deep approaches can be fine tuned with new datasets to optimize performance.We demo AFAR 1 : an open-source, deep-learning based, user-friendly tool for automated facial affect recognition.

Assessing Automated Facial Action Unit Detection Systems for Analyzing Cross-Domain Facial Expression Databases

“…Because manual FACS coding has the disadvantage of being time consuming, automatic detection of FACS AUs has been an active area of research [ 5 ]. Automated facial AU detection systems are available as both commercial tools (e.g., Affectiva, FaceReader) and open-source tools (e.g., OpenFace, [ 6 , 7 ] and Automated Facial Affect Recognition (AFAR) [ 8 , 9 ]). One study found that OpenFace and AFAR generally performed similarly, but average results were slightly better for AFAR [ 5 ].…”

“…Second, there has been no systematic comparison of AU detection accuracy among systems. FaceReader is a commercial software designed to analyze facial expressions, whereas OpenFace [ 6 , 7 ] is the dominant shareware automatic facial computing system for many applied situations [ 17 , 18 ], and AFAR is an open-source, state-of-the-art, algorithm-based user-friendly tool for automated AU detection [ 8 , 9 ]. Although comparisons of the performance of these systems are interesting for newcomers and important in terms of system selection, to our best knowledge, no studies have compared these three tools as of yet.…”

“…Here, we addressed these issues by comparing the performance of three AU detection systems: FaceReader, OpenFace [ 6 , 7 ], and AFAR [ 8 , 9 ]. We used a cross-domain database such as in-the-wild data from YouTube videos, posed facial movements expressing emotional states and social facial expressions during conversations.…”

“…Other comprehensive information such as the F1 score, negative agreements, and others are described in the Supplemental Material ( ) for the sake of transparency of the results. Moving beyond previous research [ 7 , 8 , 9 , 10 , 11 ], this study aimed to provide bias scores using AUs’ co-occurrence and predictions. We hypothesized that all systems would show above-random performance (i.e., AUC = 0.50).…”

Automated Detection of Optimal DBS Device Settings

“…The mirth response is quantified as the intensity summation of two facial actions units (AU) as defined by the Facial Action Coding System [7]: AU6 (cheek raiser) and AU12 (lip corner puller), which together make the Duchenne smile that represents and signals positive emotion [2,3,9,11]. We acquired the intensity of these 2 AUs using Automatic Facial Affect Recognition (AFAR) [14], a powerful toolkit for assessing severity of negative and positive affect [6,12,13]. We explored its effectiveness to objectively measure mirth response (positive affect) to DBS adjustments.…”