In this paper, we present an embodiment perspective on viewpoint by exploring the role of eye gaze in face-to-face conversation, in relation to and interaction with other expressive modalities. More specifically, we look into gaze patterns, as well as gaze synchronization with speech, as instruments in the negotiation of participant roles in interaction. In order to obtain fine-grained information on the different modalities under scrutiny, we used the InSight Interaction Corpus (Brône, Geert & Bert Oben. 2015. Insight Interaction: A multimodal and multifocal dialogue corpus. Language Resources and Evaluation 49, 195–214.). This multimodal video corpus consists of two- and three-party interactions (in Dutch), with head-mounted scene cameras and eye-trackers tracking all participants’ visual behavior, providing a unique ‘speaker-internal’ perspective on the conversation. The analysis of interactional sequences from the corpus (dyads and triads) reveals specific patterns of gaze distribution related to the temporal organization of viewpoint in dialogue. Different dialogue acts typically display specific gaze events at crucial points in time, as, e.g., in the case of brief gaze aversion associated with turn-holding, and shared gaze between interlocutors at the critical point of turn-taking. In addition, the data show a strong correlation and temporal synchronization between eye gaze and speech in the realization of specific dialogue acts, as shown by means of a series of cross-recurrence analyses for specific turn-holding mechanisms (e.g., verbal fillers co-occurring with brief moments of gaze aversion).

This contribution focuses on the shrug as it is used in stance taking contexts in face-to-face conversations. The shrug qualifies as a ‘compound enactment’, in which prototypically different gestures are combined: “the eye-brows (which are being raised), the hands (which are turned so that the palms face up), the forearms (which may be lifted), and the shoulders (which are also raised). In addition, the head may be tilted” (Streeck 2009: 189. Gesturecraft. The manu-facture of meaning. Amsterdam: Benjamins). More recent studies show that instead of a head tilt, a headshake can also be a part of the shrug (Schoonjans 2014. Modalpartikeln als multimodale Konstruktionen: Eine korpusbasierte Kookkurrenzanalyse von Modalpartikeln und Gestik im Deutschen. Leuven: KU Leuven dissertation). We report on an empirical study, in which the shrug or some of its gestural components are used to express obviousness on the part of the speaker. Although our data reveal multimodal patterns in the expression of obviousness, this study singles out the gestural dimension. The paper is structured as follows. Section 2 explains the methodological aspects of our study, after which Section 3 presents the formal and quantitative results, illustrated with four examples. The paper ends in Section 4 with a brief discussion of our major findings.

No abstract

Multimodal expressions of obviousness. An empirical corpus studyThis article presents a multimodal study of the expressions of obviousness in spoken Dutch. Natural conversations between two participants were studied to present an overview of the verbal and non-verbal elements that express obviousness and their interplay. Markers of obviousness are verbal elements like dus, ja, natuurlijk, gewoon and allee as well as generic and deliberately unfinished utterances (cases of so-called ‘aposiopesis’), but also co-speech gestures like the opening of the hand(s), the tilt, the shoulder shrug, raising of the eyebrows and the headshake. The interplay of these markers was studied to answer the question whether one can speak of multimodal constructions of obviousness. The study reveals intriguing multimodal co-occurrences: certain verbal elements and gestures are often used together to express the interlocutor’s stance.

Many studies describe uh(m) as a symptom of a production difficulty, a turn holding signalor an announcement of a delay in speech (Christenfeld 1994, Maclay & Osgood 1959,Clark & Fox Tree 2002). They all emphasize the role of uh(m) as a projection of furthertalk by the same speaker. However, our data show that uh(m) can also occur in turn-finalposition, thus not projecting continuation of a turn. This study compares turn-final occurrencesof uh(m) to the typical turn-medial uh(m) from a multimodal perspective. Theanalysis shows that turn-final uh(m) is only rarely preceded by a pause, more oftenaccompanied by facial expressions and co-occurs with a retraction of a hand gesture.Also, the speaker frequently gazes at the interlocutor during turn-final uh(m). These formaldifferences may be linked to a functional difference between turn-final and turn-medialuh(m).