Detecting Decision Ambiguity from Facial Images

Jahoda, Pavel; Vobecký, Antonín; Čech, Jan; Matas, Jiřı́

doi:10.1109/fg.2018.00080

Cited by 3 publications

(1 citation statement)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first is a simple multimodal neural network that separately processes video, au- dio, and text inputs and combines the features for a three-class softmax classification. The second is a detection model trained with adult data that takes visual information (Jahoda et al, 2018). This second baseline is a traditional machine-learning approach using SVMs and LBP descriptors.…”

Section: Experimental Evaluationmentioning

confidence: 99%

Learning Multimodal Cues of Children’s Uncertainty

Cheng,

Inan,

Mbarki

et al. 2023

Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue

View full text Add to dashboard Cite

Understanding uncertainty plays a critical role in achieving common ground (Clark et al., 1983). This is especially important for multimodal AI systems that collaborate with users to solve a problem or guide the user through a challenging concept. In this work, for the first time, we present a dataset annotated in collaboration with developmental and cognitive psychologists for the purpose of studying nonverbal cues of uncertainty. We then present an analysis of the data, studying different roles of uncertainty and its relationship with task difficulty and performance. Lastly, we present a multimodal machine learning model that can predict uncertainty given a real-time video clip of a participant, which we find improves upon a baseline multimodal transformer model. This work informs research on cognitive coordination between human-human and human-AI and has broad implications for gesture understanding and generation. The anonymized version of our data and code will be publicly available upon the completion of the required consent forms and data sheets.

show abstract

Section: Experimental Evaluationmentioning

confidence: 99%