Coding gestural behavior with the NEUROGES-ELAN system

Abstract

Knowledge about gestural behavior is theoretically relevant for understanding cognitive, emotional, and interactive processes, but it also has far-reaching practical implications for diagnostics and therapy in the clinical context, for learning processes, and for obtaining communicative competencies. Thus, research on gestural behavior is conducted in several academic disciplines, including neuroscience, medicine, psychology, linguistics, anthropology, and social sciences. Especially because of the shift in recent decades toward transferring information through visual media, a thorough knowledge of how gesture reflects and affects the gesturer's thinking and feeling, as well as of how it influences the recipient's cognitive and emotional processes, is becoming increasingly important.Despite many examinations carried through in various disciplines, the state of knowledge about the neuropsychology of gestural behavior has not developed far beyond the popular level (i.e., "body language"). This situation is partly due to methodological deficiencies in the analysis of gestural behavior. In many studies, gesture units and gesture types are not clearly defined, and interrater agreement is rarely examined. Furthermore, the assumptions behind the diverse gesture classifications applied in the studies are not made transparent, and sometimes there is an implicit claim that the particular coding system suits all research questions. In addition, different forms of presenting data (e.g., in terms of total numbers [n] of gesturesn/min, n/5 min, n/100 words, n/time speaking, n/time of interview, or logarithms-or of handedness-right vs. left hands only; right, left, or both hands; no differentiation of hand laterality; or handedness indices) are an obstacle for comparing the results of different studies and for gradually building up a corpus of knowledge on gestural behavior. By way of example, the apparent contradictions between study results on the relationship between aphasia (loss of the ability to produce and/or comprehend language) and gesture production/comprehension-that is, between an equally distributed deficit and a compensatory use of gestures (see, e.g., Lott, 1999)-are mainly due to a lack of scrutiny during development of the study methods.A similar situation exists in the area of media annotation, even when only the digital era is considered. A variety of tools are available to the public, each designed for a certain group of users with a specific task or a particular platform (operating system) in mind. The data produced by these tools are stored in various ways and in different formats. This is another obstacle to interoperability and comparison, in addition to those mentioned above.The XML (Extensible Markup Language) data produced by the majority of contemporary tools are a significant improvement on the old-time proprietary formats. XML data are relatively easy to transform, but nevertheless the conversion is rarely lossless. In an attempt to improve interoperability between tools from multiple modalities, ...

In Infants' Hands: Identification of Preverbal Infants at Risk for Primary Language Delay

“…Three assistants coded the videos in both realtime and frame by frame analyses for the occurrence of pointing using the annotation tool ELAN (Lausberg, H. & Sloetjes, 2009). Pointing was coded when the infant extended the hand and the arm toward an object or a picture without grabbing or Note.…”

The timing of head movements: The role of prosodic heads and edges

“…For gestures, we used ELAN annotation software, a tool that allows precise, frame-by-frame navigation through the video recording (Lausberg and Sloetjes, 2009). As Fig.…”

How do 3- and 5-year-olds respond to under- and over-informative utterances?

“…Off-line coding was conducted using the multimedia annotator ELAN, developed by the Technical Group of the Max Planck Institute for Psycholinguistics (Lausberg and Sloetjes, 2009). …”