The more the merrier: Analysing the affect of a group of people in images

Dhall, Abhinav; Joshi, Jyoti; Sikka, Karan; Goecke, Roland; Sebe, Nicu

doi:10.1109/fg.2015.7163151

Cited by 65 publications

(101 citation statements)

References 32 publications

Supporting

Mentioning

101

Contrasting

Order By: Relevance

“…In the experimental study the proposed approach (Fig. 2) was compared with the baseline support vector regression model for the CENTRIST scene descriptor [6]. For the CNN network in our pipeline (weak predictor #6) which classifies the whole group image, we used the Adagrad optimizer with learning rate of 1e-4, batch size of 40, aggressive data augmentation (random horizontal flip, ±10° of rotation, up to 10% zoom, channel shift by ±5) and trained this network for 30 epochs.…”

Section: Resultsmentioning

confidence: 99%

Group-level emotion recognition using transfer learning from face identification

Rassadin

Gruzdev

Savchenko

2017

Proceedings of the 19th ACM International Conference on Multimodal Interaction

View full text Add to dashboard Cite

In this paper we describe our algorithmic approach, which was used for submissions in the fifth Emotion Recognition in the Wild (EmotiW 2017) group-level emotion recognition sub-challenge. We extracted feature vectors of detected faces using the Convolutional Neural Network trained for face identification task, rather than traditional pre-training on emotion recognition problems. In the final pipeline an ensemble of Random Forest classifiers was learned to predict emotion score using available training set. In case when the faces have not been detected, one member of our ensemble extracts features from the whole image. During our experimental study, the proposed approach showed the lowest error rate when compared to other explored techniques. In particular, we achieved 75.4% accuracy on the validation data, which is 20% higher than the handcrafted feature-based baseline. The source code using Keras framework is publicly available.

show abstract

Section: Resultsmentioning

confidence: 99%

Group-level emotion recognition using transfer learning from face identification

Rassadin

Gruzdev

Savchenko

2017

Proceedings of the 19th ACM International Conference on Multimodal Interaction

View full text Add to dashboard Cite

show abstract

“…An interesting direction is to compute image saliency and weight the confidence of subjects who fall in the highly salient area. A natural extension of the proposed work is adding negative emotion group images to the database and framework [55]. Further, human body pose can be merged with the face analysis of a group of people.…”

Section: Discussionmentioning

confidence: 99%

Automatic Group Happiness Intensity Analysis

Dhall

Goecke

Gedeon

2015

IEEE Trans. Affective Comput.

Self Cite

View full text Add to dashboard Cite

Abstract-The recent advancement of social media has given users a platform to socially engage and interact with a larger population. Millions of images and videos are being uploaded everyday by users on the web from different events and social gatherings. There is an increasing interest in designing systems capable of understanding human manifestations of emotional attributes and affective displays. As images and videos from social events generally contain multiple subjects, it is an essential step to study these groups of people. In this paper, we study the problem of happiness intensity analysis of a group of people in an image using facial expression analysis. A user perception study is conducted to understand various attributes, which affect a person's perception of the happiness intensity of a group. We identify the challenges in developing an automatic mood analysis system and propose three models based on the attributes in the study. An 'in the wild' image-based database is collected. To validate the methods, both quantitative and qualitative experiments are performed and applied to the problem of shot selection, event summarisation and album creation. The experiments show that the global and local attributes defined in the paper provide useful information for theme expression analysis, with results close to human perception results.

show abstract

“…Our method is based on the idea that the group emotion can be inferred using both top-down [3] and bo om-up [10] approaches. e former considers the scene context, such as background, clothes, e la er estimates the face expressions of each person in the group.…”

Section: Proposed Methodsmentioning

confidence: 99%

Emotion recognition in the wild using deep neural networks and Bayesian classifiers

Surace

Patacchiola

Sönmez

et al. 2017

Proceedings of the 19th ACM International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Group emotion recognition in the wild is a challenging problem, due to the unstructured environments in which everyday life pictures are taken. Some of the obstacles for an e ective classi cation are occlusions, variable lighting conditions, and image quality. In this work we present a solution based on a novel combination of deep neural networks and Bayesian classi ers. e neural network works on a bo om-up approach, analyzing emotions expressed by isolated faces. e Bayesian classi er estimates a global emotion integrating top-down features obtained through a scene descriptor. In order to validate the system we tested the framework on the dataset released for the Emotion Recognition in the Wild Challenge 2017. Our method achieved an accuracy of 64.68% on the test set, signi cantly outperforming the 53.62% competition baseline.

show abstract

The more the merrier: Analysing the affect of a group of people in images

Cited by 65 publications

References 32 publications

Group-level emotion recognition using transfer learning from face identification

Group-level emotion recognition using transfer learning from face identification

Automatic Group Happiness Intensity Analysis

Emotion recognition in the wild using deep neural networks and Bayesian classifiers

Contact Info

Product

Resources

About