ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors

Subramanian, Ramanathan; Wache, Julia; Abadi, Mojtaba Khomami; Vieriu, Radu L.; Winkler, Stefan; Sebe, Nicu

doi:10.1109/taffc.2016.2625250

Cited by 345 publications

(251 citation statements)

References 48 publications

Supporting

Mentioning

248

Contrasting

Unclassified

Order By: Relevance

“…The only similar work that uses the 16PF methodology is presented in [23]. However, it only focuses on analyzing a set of still images using a Convolutional Neural Network (CNN) and CNN features, while our research uses the FACS methodology for studying the face, as FACS is better at predicting hidden emotions [24][25][26], hence will provide more accuracy and reliability to the personality prediction task knowing there is a close relationship between personality traits and how emotions are expressed [27,28]. FACS [29] also offers an in-depth analysis of the facial muscle activity by studying micro expressions and, as we use video recordings of subjects' frontal face and not still images, our paper shows significant prediction accuracy improvement compared to [23] which we detail in the next sections.…”

Section: Introductionmentioning

confidence: 99%

Predicting the Sixteen Personality Factors (16PF) of an individual by analyzing facial features

Gavrilescu

Vizireanu

2017

J Image Video Proc.

View full text Add to dashboard Cite

We propose a novel three-layered neural network-based architecture for predicting the Sixteen Personality Factors from facial features analyzed using Facial Action Coding System. The proposed architecture is built on three layers: a base layer where the facial features are extracted from each video frame using a multi-state face model and the intensity levels of 27 Action Units (AUs) are computed, an intermediary level where an AU activity map is built containing all AUs' intensity levels fetched from the base layer in a frame-by-frame manner, and a top layer consisting of 16 feed-forward neural networks trained via backpropagation which analyze the patterns in the AU activity map and compute scores from 1 to 10, predicting each of the 16 personality traits. We show that the proposed architecture predicts with an accuracy of over 80%: warmth, emotional stability, liveliness, social boldness, sensitivity, vigilance, and tension. We also show there is a significant relationship between the emotions elicited to the analyzed subjects and high prediction accuracy obtained for each of the 16 personality traits as well as notable correlations between distinct sets of AUs present at high-intensity levels and increased personality trait prediction accuracy. The system converges to a stable result in no more than 1 min, making it faster and more practical than the Sixteen Personality Factors Questionnaire and suitable for real-time monitoring of people's personality traits.

show abstract

Section: Introductionmentioning

confidence: 99%

Predicting the Sixteen Personality Factors (16PF) of an individual by analyzing facial features

Gavrilescu

Vizireanu

2017

J Image Video Proc.

View full text Add to dashboard Cite

show abstract

“…Content-centric approaches [17], [18] predict the likely elicited emotions by examining image, audio and videobased emotion correlates [17], [23], [25]. In contrast, usercentric AR methods [14]- [16] estimate the stimulus-evoked emotion based on physiological changes observed in viewers (content consumers). Physiological signals indicative of emotions include pupillary dilation [26], eye-gaze patterns [9], [27] and neural activity [14], [15], [28].…”

Section: Affect Recognitionmentioning

confidence: 99%

Recognition of Advertisement Emotions With Application to Computational Advertising

Shukla

Gullapuram

Katti

et al. 2022

IEEE Trans. Affective Comput.

Self Cite

View full text Add to dashboard Cite

Advertisements (ads) often contain strong affective content to capture viewer attention and convey an effective message to the audience. However, most computational affect recognition (AR) approaches examine ads via the text modality, and only limited work has been devoted to decoding ad emotions from audiovisual or user cues. This work (1) compiles an affective ad dataset capable of evoking coherent emotions across users; (2) explores the efficacy of content-centric convolutional neural network (CNN) features for AR vis-ã-vis handcrafted audio-visual descriptors; (3) examines user-centric ad AR from Electroencephalogram (EEG) responses acquired during ad-viewing, and (4) demonstrates how better affect predictions facilitate effective computational advertising as determined by a study involving 18 users. Experiments reveal that (a) CNN features outperform audiovisual descriptors for content-centric AR; (b) EEG features are able to encode ad-induced emotions better than content-based features; (c) Multi-task learning performs best among a slew of classification algorithms to achieve optimal AR, and (d) Pursuant to (b), EEG features also enable optimized ad insertion onto streamed video, as compared to contentbased or manual insertion techniques in terms of ad memorability and overall user experience.

show abstract

“…Big-five personality scales and affective self-ratings of 58 users together with their EEG, ECG, GSR, and facial activity data were included in the ASCERTAIN dataset [123] . The number of videos used as the stimulus is 36 and the length of each video clip is between 51 and 128 seconds.…”

Section: Datasets Consisting Of Videos Onlymentioning

confidence: 99%

Affective Computing for Large-scale Heterogeneous Multimedia Data

Zhao

Wang

Soleymani

et al. 2019

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

The wide popularity of digital photography and social networks has generated a rapidly growing volume of multimedia data (i.e., image, music, and video), resulting in a great demand for managing, retrieving, and understanding these data. Affective computing (AC) of these data can help to understand human behaviors and enable wide applications. In this article, we survey the state-of-the-art AC technologies comprehensively for large-scale heterogeneous multimedia data. We begin this survey by introducing the typical emotion representation models from psychology that are widely employed in AC. We briefly describe the available datasets for evaluating AC algorithms. We then summarize and compare the representative methods on AC of different multimedia types, i.e., images, music, videos, and multimodal data, with the focus on both handcrafted features-based methods and deep learning methods. Finally, we discuss some challenges and future directions for multimedia affective computing.

show abstract

ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors

Cited by 345 publications

References 48 publications

Predicting the Sixteen Personality Factors (16PF) of an individual by analyzing facial features

Predicting the Sixteen Personality Factors (16PF) of an individual by analyzing facial features

Recognition of Advertisement Emotions With Application to Computational Advertising

Affective Computing for Large-scale Heterogeneous Multimedia Data

Contact Info

Product

Resources

About