Emotion recognition using facial and audio features

Abstract

Human Computer Interaction is an upcoming scientific field which aims at inter-communication between humans and computers. A major element of this field is Human Emotion Recognition. The most expressive way humans display emotions is through facial expressions. Traditionally, emotion recognition has been performed on laboratory controlled data. While undoubtedly worthwhile at the time, such lab controlled data poorly represents the environment and conditions faced in realworld situations. With the increase in the number of video clips online, it is worthwhile to explore the performance of emotion recognition methods that work 'in the wild' .This work mainly focuses on automatic emotion recognition in a wild video sample. In this task, we have worked on the problem of human emotion recognition using a combination of video features and audio features. The technique that we have utilized for emotion detection involves a blend of Optical flow, Gabor Filtering, few other facial features and audio features. Training and Classification is performed using Support Vector MachineHidden Markov Model (HMM). The unique thing about our methodology is that it produces better results for some particular class of emotions as compared to the baseline score in the case of wild emotion dataset with an overall accuracy of 20.51% on the test set.

Quantifying the efficacy of an automated facial coding software using videos of parents

“…In the context of naturalistic observations, Gómez Jáuregui and Martin (2013) applied FaceReader to analyse a dataset of acted facial expressions under uncontrolled conditions and found that the software could not accurately classify any expression. Contrasting with this result, Krishna et al (2013) achieved an expression classification accuracy of 20.51% using different automated methods with the same dataset, raising further questions about the performance of FaceReader in real-life recordings. Collectively, these studies contribute to the growing body of literature on the validity, limitations, and applications of AFC in understanding emotional facial expressions in diverse contexts, emphasising the importance of advancing AFC models to effectively capture and interpret facial expressions in realistic settings.…”

Improved Spatiotemporal Local Monogenic Binary Pattern for Emotion Recognition in The Wild

“…Some attempts were made to use other appearance-based features, e.g. Local Gabor binary pattern from three orthogonal planes (LGBP-TOP) [2], optical flow and Gabor [17]. In addition, the geometric features were only used in [17].…”

“…Local Gabor binary pattern from three orthogonal planes (LGBP-TOP) [2], optical flow and Gabor [17]. In addition, the geometric features were only used in [17]. However, it is observed that the combination of optical flow, Gabor and geometric features work poorer than the baseline.…”