Automatic stress detection evaluating models of facial action units

Giannakakis, Giorgos; Koujan, Mohammad Rami; Roussos, Anastasios; Marias, Kostas

doi:10.1109/fg47880.2020.00129

Cited by 20 publications

(16 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our results seem to be corroborated by other studies. A study using AUs and SVM found that AU17, AU23, and AU25 intensities are modulated by stress conditions [10]. So, it might be the case that decision-making processes trigger a response similar to stressful stimuli [28].…”

Section: Resultsmentioning

confidence: 99%

“…Given all this, it is perhaps unsurprising that action units have been used as input for machine learning models. For example, a relatively simple support vector machine (SVM) reached 0.75 accuracy when using AUs as input for automatic stress detection [10]. SVM and k-nearest neighbors (KNN) algorithms can classify expressions of "pain" vs "no pain" and even their intensity [17,23].…”

Section: Facial Expressions and Machine Learningmentioning

confidence: 99%

“…The AUs used for this work were extracted using OpenFace [2]. OpenFace extracts 17 action units (1,2,4,5,6,7,9,10,12,14,15,17,20,23,25,26, and 45) that can be described either in terms of their presence (0 or 1) or in terms of their intensity (from 0 to 5). In our work, we extracted just the intensity information since, by itself, it can provide a number ranging from 0, the absence of the activity in the AUs, to 5, conveying the maximum intensity in the AUs.…”

Section: Data Extractionmentioning

confidence: 99%

See 2 more Smart Citations

Using Deep Learning to Detect Facial Markers of Complex Decision Making

Guglielmo

Peradejordi

Klincewicz

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In this paper, we report on an experiment with The Walking Dead (TWD), which is a narrative-driven adventure game where players have to survive in a post-apocalyptic world filled with zombies. We used OpenFace software to extract action unit (AU) intensities of facial expressions characteristic of decision-making processes and then we implemented a simple convolution neural network (CNN) to see which AUs are predictive of decision-making. Our results provide evidence that the pre-decision variations in action units 17 (chin raiser), 23 (lip tightener), and 25 (parting of lips) are predictive of decision-making processes. Furthermore, when combined, their predictive power increased up to 0.81 accuracy on the test set; we offer speculations about why it is that these particular three AUs were found to be connected to decision-making. Our results also suggest that machine learning methods in combination with video games may be used to accurately and automatically identify complex decision-making processes using AU intensity alone. Finally, our study offers a new method to test specific hypotheses about the relationships between higher-order cognitive processes and behavior, which relies on both narrative video games and easily accessible software, like OpenFace.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Facial Expressions and Machine Learningmentioning

confidence: 99%

Section: Data Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Using Deep Learning to Detect Facial Markers of Complex Decision Making

Guglielmo

Peradejordi

Klincewicz

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…After discovering breathing patterns through temperature changes near the nostrils, twodimensional respiration variability spectrogram sequences were constructed using these data and were used to recognize stress. Giannakakis et al [37] recognized stress based on facial action unit information obtained from nonrigid 3D facial landmarks, the histogram of oriented gradients (HOG), and the SVM. The limitations of the aforementioned methods are that they cannot utilize the changes in the facial colors and the full facial image because the entire image information is not used.…”

Section: Facial-action-unit-based Stress Recognition Methodsmentioning

confidence: 99%

Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information

Jeon

Bae

Lee

et al. 2021

Sensors

View full text Add to dashboard Cite

In recent times, as interest in stress control has increased, many studies on stress recognition have been conducted. Several studies have been based on physiological signals, but the disadvantage of this strategy is that it requires physiological-signal-acquisition devices. Another strategy employs facial-image-based stress-recognition methods, which do not require devices, but predominantly use handcrafted features. However, such features have low discriminating power. We propose a deep-learning-based stress-recognition method using facial images to address these challenges. Given that deep-learning methods require extensive data, we constructed a large-capacity image database for stress recognition. Furthermore, we used temporal attention, which assigns a high weight to frames that are highly related to stress, as well as spatial attention, which assigns a high weight to regions that are highly related to stress. By adding a network that inputs the facial landmark information closely related to stress, we supplemented the network that receives only facial images as the input. Experimental results on our newly constructed database indicated that the proposed method outperforms contemporary deep-learning-based recognition methods.

show abstract

“…Recovering the 3D geometry of human faces from monocular images is a challenging problem that has attracted much attention due to its major role in many applications, ranging from facial reenactment, performance capture and tracking [16], facial expression recognition [17], [18], etc. Owing to their pose and illumination invariance, 3D facial data constitute an indispensable geometrical description of faces for various facial image processing systems.…”

Section: D Face Reconstructionmentioning

confidence: 99%