Evaluating Speech, Face, Emotion and Body Movement Time-series Features for Automated Multimodal Presentation Scoring

Ramanarayanan, Vikram; Leong, Chee Wee; Chen, Lei; Feng, Ganjun; Suendermann-Oeft, David

doi:10.1145/2818346.2820765

Cited by 40 publications

(17 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent developments in multimodal data capture techniques have made it possible to detect these behavioral expressions during public speaking situations to assess the performance of participants. Ramanarayanan et al 21 extracted several nonverbal behavioral cues (head pose, gaze, and facial expressions) and speaking proficiency features (prosody, pronunciation and grammar) from 17 participants during five public speaking tasks. The authors correlated these extracted features with human-rated scores to find the modalities that are useful in predicting public speaking performance.…”

Section: Psychological Stress and Performance Assessment In Public Speaking Situationsmentioning

confidence: 99%

Design and evaluation of postural interactions between users and a listening virtual agent during a simulated job interview

Jáuregui

Giraud

Isableu

et al. 2021

Computer Animation & Virtual

View full text Add to dashboard Cite

show abstract

Section: Psychological Stress and Performance Assessment In Public Speaking Situationsmentioning

confidence: 99%

Design and evaluation of postural interactions between users and a listening virtual agent during a simulated job interview

Jáuregui

Giraud

Isableu

et al. 2021

Computer Animation & Virtual

View full text Add to dashboard Cite

show abstract

“…We recently analyzed (Chen, Leong, Feng, Lee, & Somasundaran, 2015;Chen et al, 2014;Ramanarayanan, Chen, Leong, Feng, & Suendermann-Oeft, 2015) how fusing features obtained from different multimodal data streams, such as speech, face, body movement, and emotion tracks, can be applied to the scoring of multimodal presentations. We analyzed multimodal data collected by Chen et al (2015) consisting of synchronized and preprocessed recordings from 56 sessions involving speakers giving presentations on different topics.…”

Section: Multimodal Assessmentmentioning

confidence: 99%

“…We further observed that these features allowed us to achieve a prediction performance better than the human interrater agreement (as measured by the correlation between scores provided by two nonexpert human raters) for a subset of these scores. For further details, please see Ramanarayanan, Chen, et al ().…”

Section: Multimodal Assessmentmentioning

confidence: 99%

A Multimodal Dialog System for Language Assessment: Current State and Future Directions

Suendermann-Oeft

Ramanarayanan

et al. 2017

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

We present work in progress on a multimodal dialog system for English language assessment using a modular cloud‐based architecture adhering to open industry standards. Among the modules being developed for the system, multiple modules heavily exploit machine learning techniques, including speech recognition, spoken language proficiency rating, speaker recognition, and the scoring of behaviors in multimodal data streams.

show abstract

“…Emotions can also be measured through interaction with gesture, gaze and auditory stimuli. For instance, Ramanarayanan et al () used a variety of equipment and software tools including Microsoft Kinect to evaluate the quality of public presentations in relation to speech, face, emotion and body movement. Lu and Petiot () used a set of auditory stimuli to convey and assess a set of emotions such as funny, serious, relaxed, and depressed.…”

Section: Introductionmentioning

confidence: 99%

Beyond Happy‐or‐Not: Using Emoji to Capture Visitors’ Emotional Experience

Angeli¹,

Kelly²,

O’Neill³

2020

Curator The Museum Journal

View full text Add to dashboard Cite

Museums are emotionally driven sites. People visit museums to feel and their emotions influence how the museum and its artefacts are perceived. Thus, evaluating emotional states are increasingly important for museums. However, evaluating visitors' experiences is increasingly challenging, especially with the introduction of new and emerging technology. Moreover, people's behaviour is not strictly objective and rational. While emotional states are subjective and hard to verbalize or observe, emoji are often used to express emotions on mobile and smartphone messaging applications. In this paper we investigate whether emoji can capture emotional states elicited by museum experiences, supporting traditional methods such as interviews. While other non-verbal self-report methods have been used to evaluate emotions, this is the first tool of this kind designed specifically to measure emotions elicited by museum experiences. We designed a set of 9 emoji illustrating a variety of emotional states beyond happyor-not. Then, we confirmed that participants understood our emoji's intended concept using a word association task. Finally, we used our 9 emoji to evaluate an interactive museum experience. We also run interviews and we investigated the correspondence between participants' comments and the emoji they chose. Through this study we gained a better understanding of how the emoji can be deployed to capture a range of visitors' emotional experiences. Our findings suggest that emoji can capture which emotional states participants felt beyond the happy-or-not dichotomy, but that they should be complemented with traditional methods such as interviews to understand why specific emotions were felt.

show abstract

Evaluating Speech, Face, Emotion and Body Movement Time-series Features for Automated Multimodal Presentation Scoring

Cited by 40 publications

References 13 publications

Design and evaluation of postural interactions between users and a listening virtual agent during a simulated job interview

Design and evaluation of postural interactions between users and a listening virtual agent during a simulated job interview

A Multimodal Dialog System for Language Assessment: Current State and Future Directions

Beyond Happy‐or‐Not: Using Emoji to Capture Visitors’ Emotional Experience

Contact Info

Product

Resources

About