Machine Analysis of Facial Expressions

Pantić, Maja; Bartlett,

doi:10.5772/4847

Cited by 176 publications

(160 citation statements)

References 123 publications

Supporting

Mentioning

147

Contrasting

Unclassified

Order By: Relevance

“…Facial expressions [20], [21], [22], vocal features [23] [24] [25], body movements and postures [26], [27], [11], [28], physiological signals [29] have been used as inputs during these attempts, although multimodal emotion recognition is currently gaining ground [7], [30], [31], [32], [33]. Nevertheless, most of the work has considered the integration of information from facial expressions and speech [34], [35] and there have been relatively few attempts to combine information from body movement and gestures in a multimodal framework.…”

Section: Related Workmentioning

confidence: 99%

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Kessous¹,

Castellano

2009

J Multimodal User Interfaces

200

View full text Add to dashboard Cite

In this paper a study on multimodal automatic emotion recognition during a speech-based interaction is presented. A database was constructed consisting of people pronouncing a sentence in a scenario where they interacted with an agent using speech. Ten people pronounced a sentence corresponding to a command while making 8 different emotional expressions. Gender was equally represented, with speakers of several different native languages including French, German, Greek and Italian. Facial expression, gesture and acoustic analysis of speech were used to extract features relevant to emotion. For the automatic classification of unimodal data, bimodal data and multimodal data, a system based on a Bayesian classifier was used. After performing an automatic classification of each modality, the different modalities were combined using a multimodal approach. Fusion of the modalities at the feature level (before running the classifier) and at the results level (combining results from classifier from each modality) were compared. Fusing the multimodal data resulted in a large increase in the recognition rates in comparison to the unimodal systems: the multimodal approach increased the recognition rate by more than 10% when compared to the most successful unimodal system. Bimodal emotion recognition based on all combinations of the modalities (i.e., 'face-gesture', 'facespeech' and 'gesture-speech') was also investigated. The results show that the best pairing is 'gesture-speech'. Using all three modalities resulted in a 3.3% classification improvement over the best bimodal results.

show abstract

Section: Related Workmentioning

confidence: 99%

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Kessous¹,

Castellano

2009

J Multimodal User Interfaces

200

View full text Add to dashboard Cite

show abstract

“…Important sources of information for facial representation are image discontinuities, such as furrows and wrinkles [27,39]. Representations used for facial affect recognition are often categorized as global and local representations.…”

Section: Introductionmentioning

confidence: 99%

Local Zernike Moment Representation for Facial Affect Recognition

Sarıyanidi¹,

Güneş²,

Gokmen³

et al. 2013

Procedings of the British Machine Vision Conference 2013

View full text Add to dashboard Cite

In this paper, we propose to use local Zernike Moments (ZMs) for facial affect recognition and introduce a representation scheme based on performing non-linear encoding on ZMs via quantization. Local ZMs provide a useful and compact description of image discontinuities and texture. We demonstrate the use of this ZM-based representation for posed and discrete as well as naturalistic and continuous affect recognition on standard datasets, and show that ZM-based representations outperform well-established alternative approaches for both tasks. To the best of our knowledge, the performance we achieved on CK+ dataset is superior to all results reported to date.

show abstract

“…Considering instead the channel visual, emotions arise from the following aspects: facial expressions, movements (actions) facial movements and postures of the body (which may be less susceptible to masking and inconsistency). Most of the work on the analysis and recognition of emotions is based on the detection of facial expressions, addressing two main approaches (Cohn, 2006;Pantic & Bartlett, 2007): the recognition based on elementary units of facial muscles action (AU), that are part of the coding system of facial expression called the Facial Action coding -FACS (Ekman & Friesen 1977), and recognition based on spatial and temporal characteristics of the face. FACS is a system used for measuring all visually distinguishable facial movements in terms of atomic actions called Facial Action Unit (AU).…”

Section: Detection Of Human Emotionsmentioning

confidence: 99%

Affective Human-Humanoid Interaction Through Cognitive Architecture

Infantino¹

2012

The Future of Humanoid Robots - Research and Applications

View full text Add to dashboard Cite

Machine Analysis of Facial Expressions

Cited by 176 publications

References 123 publications

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Local Zernike Moment Representation for Facial Affect Recognition

Affective Human-Humanoid Interaction Through Cognitive Architecture

Contact Info

Product

Resources

About