On the possibility of predicting gaze aversion to improve video-chat efficiency

Ward, Nigel; Jurado, Chelsey N.; Garcı́a, Ricardo; Ramos, Florencia A.

doi:10.1145/2857491.2857497

Cited by 4 publications

(1 citation statement)

References 24 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The research into how speech and eye gaze are linked lead to a better understanding of the underlying cognitive mechanisms, but also this relation has been studied for practical applications in Educational Science (e.g., Jarodzka et al, 2017), human robot interaction (e.g., Chidambaram et al, 2012;Ham et al, 2015), web-based conferencing (e.g., Ward et al, 2016), and virtual reality (VR) systems (e.g., Garau et al, 2003;Batrinca et al, 2013). Some of those studies hold under operational assumptions such as simulating gaze aversion through head movements alone, conducting research under highly controlled conditions, which does not reflect real-life settings, or encoding just the presence of human speech rather than exhaustive speech analysis.…”

Section: The Present Studymentioning

confidence: 99%

Speech Driven Gaze in a Face-to-Face Interaction

2021

View full text Add to dashboard Cite

Gaze and language are major pillars in multimodal communication. Gaze is a non-verbal mechanism that conveys crucial social signals in face-to-face conversation. However, compared to language, gaze has been less studied as a communication modality. The purpose of the present study is 2-fold: (i) to investigate gaze direction (i.e., aversion and face gaze) and its relation to speech in a face-to-face interaction; and (ii) to propose a computational model for multimodal communication, which predicts gaze direction using high-level speech features. Twenty-eight pairs of participants participated in data collection. The experimental setting was a mock job interview. The eye movements were recorded for both participants. The speech data were annotated by ISO 24617-2 Standard for Dialogue Act Annotation, as well as manual tags based on previous social gaze studies. A comparative analysis was conducted by Convolutional Neural Network (CNN) models that employed specific architectures, namely, VGGNet and ResNet. The results showed that the frequency and the duration of gaze differ significantly depending on the role of participant. Moreover, the ResNet models achieve higher than 70% accuracy in predicting gaze direction.

show abstract