Knowledge about gestural behavior is theoretically relevant for understanding cognitive, emotional, and interactive processes, but it also has far-reaching practical implications for diagnostics and therapy in the clinical context, for learning processes, and for obtaining communicative competencies. Thus, research on gestural behavior is conducted in several academic disciplines, including neuroscience, medicine, psychology, linguistics, anthropology, and social sciences. Especially because of the shift in recent decades toward transferring information through visual media, a thorough knowledge of how gesture reflects and affects the gesturer's thinking and feeling, as well as of how it influences the recipient's cognitive and emotional processes, is becoming increasingly important.Despite many examinations carried through in various disciplines, the state of knowledge about the neuropsychology of gestural behavior has not developed far beyond the popular level (i.e., "body language"). This situation is partly due to methodological deficiencies in the analysis of gestural behavior. In many studies, gesture units and gesture types are not clearly defined, and interrater agreement is rarely examined. Furthermore, the assumptions behind the diverse gesture classifications applied in the studies are not made transparent, and sometimes there is an implicit claim that the particular coding system suits all research questions. In addition, different forms of presenting data (e.g., in terms of total numbers [n] of gesturesn/min, n/5 min, n/100 words, n/time speaking, n/time of interview, or logarithms-or of handedness-right vs. left hands only; right, left, or both hands; no differentiation of hand laterality; or handedness indices) are an obstacle for comparing the results of different studies and for gradually building up a corpus of knowledge on gestural behavior. By way of example, the apparent contradictions between study results on the relationship between aphasia (loss of the ability to produce and/or comprehend language) and gesture production/comprehension-that is, between an equally distributed deficit and a compensatory use of gestures (see, e.g., Lott, 1999)-are mainly due to a lack of scrutiny during development of the study methods.A similar situation exists in the area of media annotation, even when only the digital era is considered. A variety of tools are available to the public, each designed for a certain group of users with a specific task or a particular platform (operating system) in mind. The data produced by these tools are stored in various ways and in different formats. This is another obstacle to interoperability and comparison, in addition to those mentioned above.The XML (Extensible Markup Language) data produced by the majority of contemporary tools are a significant improvement on the old-time proprietary formats. XML data are relatively easy to transform, but nevertheless the conversion is rarely lossless. In an attempt to improve interoperability between tools from multiple modalities, ...

Interoperable annotation formats are fundamental to the utility, expansion, and sustainability of collective data repositories. In language development research, shared annotation schemes have been critical to facilitating the transition from raw acoustic data to searchable, structured corpora. Current schemes typically require comprehensive and manual annotation of utterance boundaries and orthographic speech content, with an additional, optional range of tags of interest. These schemes have been enormously successful for datasets on the scale of dozens of recording hours but are untenable for long-format recording corpora, which routinely contain hundreds to thousands of audio hours. Long-format corpora would benefit greatly from (semi-)automated analyses, both on the earliest steps of annotation-voice activity detection, utterance segmentation, and speaker diarization-as well as later steps-e.g., classification-based codes such as child-vsadult-directed speech, and speech recognition to produce phonetic/orthographic representations. We present an annotation workflow specifically designed for long-format corpora which can be tailored by individual researchers and which interfaces with the current dominant scheme for short-format recordings. The workflow allows semi-automated annotation and analyses at higher linguistic levels. We give one example of how the workflow has been successfully implemented in a large crossdatabase project.

No abstract