Eye-gaze experiments for conversation monitoring

Jokinen, Kristiina; Nishida, Masafumi; Yamamoto, Seiichi

doi:10.1145/1667780.1667843

Cited by 23 publications

(17 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All 29 studies used video recording to capture eye gaze during conversation, however nine did not specify how many cameras were used (Beattie, 1978 , 1979 ; Rutter et al, 1978 ; Goodwin, 1980 ; Harrigan, 1985 ; Egbert, 1996 ; Lerner, 2003 ; Park, 2015 ; Blythe et al, 2018 ). Seven studies used one camera for each participant (Lamb, 1981 ; Bavelas et al, 2002 ; Eberhard and Nicholson, 2010 ; Cummins, 2012 ; Ho et al, 2015 ; Holler and Kendrick, 2015 ; Ijuin et al, 2018 ), three studies used one camera for the whole group interaction (Kendon, 1967 ; Harrigan and Steffen, 1983 ; Streeck, 2014 ), seven studies video recorded both each participant plus the whole group interaction (Kalma, 1992 ; Novick et al, 1996 ; Brône et al, 2017 ; Kendrick and Holler, 2017 ; Auer, 2018 ; Weiss, 2018 ; Zima et al, 2019 ), two studies only video recorded two out of three participants and eye tracked the third participant (Jokinen et al, 2009 , 2013 ), one study used two cameras to capture interactions in Italian language and only one camera to capture interactions in Tzeltal and Yeli Dnye languages Rossano et al ( 2009 ). Eleven studies used camera-based eye tracking technology ( Table 4 ), which permits investigators to measure participant's visual behavior by detecting and tracking movement of different parts of the eye (see review: Morimoto and Mimica, 2005 ).…”

Section: Resultsmentioning

confidence: 99%

“…Eleven studies used camera-based eye tracking technology ( Table 4 ), which permits investigators to measure participant's visual behavior by detecting and tracking movement of different parts of the eye (see review: Morimoto and Mimica, 2005 ). Of these, two studies used a single table eye tracker to track one out of three participants (Jokinen et al, 2009 , 2013 ) and one study tracked the eyes of two out of three participants in the conversation due to technical issues (Auer, 2018 ).…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

The Role of Eye Gaze in Regulating Turn Taking in Conversations: A Systematized Review of Methods and Findings

Degutyte

Astell

2021

Front. Psychol.

View full text Add to dashboard Cite

Eye gaze plays an important role in communication but understanding of its actual function or functions and the methods used to elucidate this have varied considerably. This systematized review was undertaken to summarize both the proposed functions of eye gaze in conversations of healthy adults and the methodological approaches employed. The eligibility criteria were restricted to a healthy adult population and excluded studies that manipulated eye gaze behavior. A total of 29 articles—quantitative, qualitative and mixed methods were returned, with a wide range of methodological designs. The main areas of variability related to number of conversants, their familiarity and status, conversation topic, data collection tools—video and eye tracking—and definitions of eye gaze. The findings confirm that eye gaze facilitates turn yielding, plays a role in speech monitoring, prevents and repairs conversation breakdowns and facilitates intentional and unintentional speech interruptions. These findings were remarkably consistent given the variability in methods across the 29 articles. However, in relation to turn initiation, the results were less consistent, requiring further investigation. This review provides a starting point for future studies to make informed decisions about study methods for examining eye gaze and selecting variables of interest.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

The Role of Eye Gaze in Regulating Turn Taking in Conversations: A Systematized Review of Methods and Findings

Degutyte

Astell

2021

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…Furthermore, the use of different multimodal signals in communication becomes understandable, as it is necessary to use all available means to manage interaction. For instance, Jokinen et al [2009] noticed that head turning is relevant for turn management in multiparty conversations. Although gaze is an important initial signal of who could be next speaker, head movement can function as a more visible signal of the speaker's focus of attention, and thus willingness to take the turn or to give the turn.…”

Section: Interaction In Multiparty Conversationsmentioning

confidence: 99%

“…Another study [Jokinen et al 2009] showed that if the speaker wants to give the turn, she looks at the listeners, while the listeners tend to look at the current speaker, but turn their gaze away if they do not want to take the turn. If the listener wants to take the turn, the listener also looks at the speaker, and turn taking is agreed by the mutual gaze.…”

Section: Turn Taking Signalsmentioning

confidence: 99%

Gaze and turn-taking behavior in casual conversational interactions

Jokinen

Furukawa

Nishida

et al. 2013

ACM Trans. Interact. Intell. Syst.

Self Cite

View full text Add to dashboard Cite

Eye gaze is an important means for controlling interaction and coordinating the participants' turns smoothly. We have studied how eye gaze correlates with spoken interaction and especially focused on the combined effect of the speech signal and gazing to predict turn taking possibilities. It is well known that mutual gaze is important in the coordination of turn taking in two-party dialogs, and in this article, we investigate whether this fact also holds for three-party conversations. In group interactions, it may be that different features are used for managing turn taking than in two-party dialogs. We collected casual conversational data and used an eye tracker to systematically observe a participant's gaze in the interactions. By studying the combined effect of speech and gaze on turn taking, we aimed to answer our main questions: How well can eye gaze help in predicting turn taking? What is the role of eye gaze when the speaker holds the turn? Is the role of eye gaze as important in three-party dialogs as in two-party dialogue? We used Support Vector Machines (SVMs) to classify turn taking events with respect to speech and gaze features, so as to estimate how well the features signal a change of the speaker or a continuation of the same speaker. The results confirm the earlier hypothesis that eye gaze significantly helps in predicting the partner's turn taking activity, and we also get supporting evidence for our hypothesis that the speaker is a prominent coordinator of the interaction space. Such a turn taking model could be used in interactive applications to improve the system's conversational performance.

show abstract

“…Other attempts to employ gaze as an indicator of conversation characteristics include turn-alignment and conversation monitoring studies of Jokinen et al [6,9,8] and a recent study of Hradis et al [4] on voice activity detection during multiparty conversation.…”

Section: Level Of Engagement and Gaze In Conversationmentioning

confidence: 99%

Gaze and conversational engagement in multiparty video conversation

Bednarik

Eivazi

Hradiš

2012

Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction

View full text Add to dashboard Cite

When using a multiparty video mediated system, interacting participants assume a range of various roles and exhibit behaviors according to how engaged in the communication they are. In this paper we focus on estimation of conversational engagement from gaze signal. In particular, we present an annotation scheme for conversational engagement, a statistical analysis of gaze behavior across varying levels of engagement, and we classify vectors of computed eye tracking measures. The results show that in 74% of cases the level of engagement can be correctly classified into either high or low level. In addition, we describe the nuances of gaze during distinct levels of engagement.

show abstract

Eye-gaze experiments for conversation monitoring

Cited by 23 publications

References 10 publications

The Role of Eye Gaze in Regulating Turn Taking in Conversations: A Systematized Review of Methods and Findings

The Role of Eye Gaze in Regulating Turn Taking in Conversations: A Systematized Review of Methods and Findings

Gaze and turn-taking behavior in casual conversational interactions

Gaze and conversational engagement in multiparty video conversation

Contact Info

Product

Resources

About