ChaLearn LAP 2016: First Round Challenge on First Impressions - Dataset and Results

Ponce-López, Víctor; Chen, Baiyu; Oliu, Marc; Corneanu, Ciprian A.; Clapés, Albert; Guyon, Isabelle; Baró, Xavier; Escalante, Hugo Jair; Escalera, Sérgio

doi:10.1007/978-3-319-49409-8_32

Cited by 140 publications

(164 citation statements)

References 42 publications

Supporting

Mentioning

154

Contrasting

Order By: Relevance

“…Inherited from them, most of latest face recognition methods consider the task as a multi-class classification problem and train deep face features on large public datasets such as LFW [17], VGG-Face [10] or FaceNet [18]. While it has been shown that the trained representations are, to some extent, transferable between face recognition and affective computing [3,19], a direct application of shared CNN representations trained for both emotion and personality without large-scale datasets encompassing both emotion and personality annotations is rarely studied. Inspired by the recent advances in face recognition achieved by lightstructured networks [20], we introduce PersEmoN with a SphereFace [20] based network backbone to show thatsuch a strategy is advantageous.…”

Section: Deep Learning For Face Recognitionmentioning

confidence: 99%

“…2. For personality, we use the ChaLearn personality dataset [3], which consists of 10k short video clips with 41.6 hours (4.5M frames) in total. In this dataset, people face and speak to the camera.…”

Section: Dataset and Evaluation Protocolmentioning

confidence: 99%

“…To assess the quality of emotion predictions from our PersEmoN, we calculate the mean square errors (MSEs) between the predicted values of personality traits and ground truth. For the evaluation of the personality recognition, we apply two metrics used in ECCV 2016 ChaLearn First Impression Challenge [3], namely mean accuracy A and coefficient of determination R 2 , which are defined as TABLE 1 Results (MSE) of emotion task on Aff-Wild.…”

Section: Dataset and Evaluation Protocolmentioning

confidence: 99%

“…In this paper we consider the Big Five personality traits (Extraversion, Agreeableness, Conscientiousness, Neuroticism and Openness) [3]. Instead of classifying pre-defined emotion categories, we use a finer-grained representation based on the circumplex model [4], in which emotions are distributed in a two-dimensional circular space spanned by the dimensions of arousal and valence.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

Zhang

Peng

Winkler

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Personality and emotion are both central to affective computing. Existing works solve them individually. In this paper we investigate if such high-level affect traits and their relationship can be jointly learned from face images in the wild. To this end, we introduce PersEmoN, an end-to-end trainable and deep Siamese-like network which we call emotion network and personality network, respectively. It consists of two convolutional network branches, one for emotion and the other for apparent personality. Both networks share their bottom feature extraction module and are optimized within a multi-task learning framework. Emotion and personality networks are dedicated to their own annotated dataset. An adversarial-like loss function is further employed to promote representation coherence among heterogeneous dataset sources. Based on this, the emotion-to-personality relationship is also well explored. Extensive experiments are provided to demonstrate the effectiveness of PersEmoN.

show abstract

Section: Deep Learning For Face Recognitionmentioning

confidence: 99%

Section: Dataset and Evaluation Protocolmentioning

confidence: 99%

Section: Dataset and Evaluation Protocolmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

Zhang

Peng

Winkler

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…In this work, we investigate AGC analysis from an early prediction perspective. This can also be viewed as a first impression of a group's cohesion, similar to the early personality assessment [12] problem in affective computing. The main contributions of this paper are as follows:…”

Section: Introductionmentioning

confidence: 99%

Predicting Group Cohesiveness in Images

Ghosh

Dhall

Sebe

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

The cohesiveness of a group is an essential indicator of the emotional state, structure and success of a group of people. We study the factors that influence the perception of grouplevel cohesion and propose methods for estimating the humanperceived cohesion on the group cohesiveness scale. In order to identify the visual cues (attributes) for cohesion, we conducted a user survey. Image analysis is performed at a group-level via a multi-task convolutional neural network. For analyzing the contribution of facial expressions of the group members for predicting the Group Cohesion Score (GCS), a capsule network is explored. We add GCS to the Group Affect database and propose the 'GAF-Cohesion database'. The proposed model performs well on the database and is able to achieve near human-level performance in predicting a group's cohesion score. It is interesting to note that group cohesion as an attribute, when jointly trained for group-level emotion prediction, helps in increasing the performance for the later task. This suggests that group-level emotion and cohesion are correlated.

show abstract

Adaptive information fusion network for multi‐modal personality recognition

Bao,

Liu,

et al. 2024

Computer Animation & Virtual

View full text Add to dashboard Cite

Personality recognition is of great significance in deepening the understanding of social relations. While personality recognition methods have made significant strides in recent years, the challenge of heterogeneity between modalities during feature fusion still needs to be solved. This paper introduces an adaptive multi‐modal information fusion network (AMIF‐Net) capable of concurrently processing video, audio, and text data. First, utilizing the AMIF‐Net encoder, we process the extracted audio and video features separately, effectively capturing long‐term data relationships. Then, adding adaptive elements in the fusion network can alleviate the problem of heterogeneity between modes. Lastly, we concatenate audio‐video and text features into a regression network to obtain Big Five personality trait scores. Furthermore, we introduce a novel loss function to address the problem of training inaccuracies, taking advantage of its unique property of exhibiting a peak at the critical mean. Our tests on the ChaLearn First Impressions V2 multi‐modal dataset show partial performance surpassing state‐of‐the‐art networks.

show abstract

ChaLearn LAP 2016: First Round Challenge on First Impressions - Dataset and Results

Cited by 140 publications

References 42 publications

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

Predicting Group Cohesiveness in Images

Adaptive information fusion network for multi‐modal personality recognition

Contact Info

Product

Resources

About