From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology

Abstract

Informal social interaction is the primordial home of human language. Linguistically diverse conversational corpora are an important and largely untapped resource for computational linguistics and language technology. Through the efforts of a worldwide language documentation movement, such corpora are increasingly becoming available. We show how interactional data from 63 languages (26 families) harbours insights about turn-taking, timing, sequential structure and social action, with implications for language technology, natural language understanding, and the design of conversational interfaces. Harnessing linguistically diverse conversational corpora will provide the empirical foundations for flexible, localizable, humane language technologies of the future.

Bottom-up discovery of structure and variation in response tokens (‘backchannels’) across diverse languages

“…For the quantitative and inductive analysis that we envision, we need relatively large and maximally diverse language resources with time-aligned transcriptions. Rather than working with noninteractive data sources or collecting new data, here we explore the potential of language resources collected by the global language documentation movement [28,4]. We curate corpora of unscripted conversation made available in language documentation archives.…”

“…In particular, we look for items that: (i) feature in the top decile of frequency counts by turn format per corpus; and (ii) occur at least once in a series of at least two produced by the same speaker. These search criteria reflect two basic observations about response tokens: their high frequency in naturally occurring talk [4], and the fact that they often occur in series of consecutive response tokens [33].…”

“…The sheer frequency and functional importance of these items means we can think of them as interactional tools: linguistic devices that accomplish social actions. Progress towards interactive language technologies hinges on understanding and modelling such interactional tools [3,4]. While some of them have long been studied in a small number of well-resourced languages [5,6], their structure and variation across diverse languages is mostly uncharted territory.…”

Turn-timing in conversations between autistic adults: typical short-gap transitions are preferred, but not achieved instantly

“…Most turns in spoken conversation are short and most transitions between turns consist of very short gaps between speakers [4], which are preferred to other possible kinds of transition such as longer gaps or overlaps (two speakers talking at once). A modal value of around 200 milliseconds of silence between speakers has been shown for a wide range of languages and speakers, with only minor languagespecific variations [5][6][7][8].…”

Backchannels in conversations between autistic adults are less frequent and less diverse prosodically and lexically

“…Previous work has also shown that the rate of BCs produced and, more importantly, their specific lexical and intonational realisation, can have a profound influence on (perceived) communicative success and mutual understanding, as well as on subjective judgements by conversational partners. This has been explored both in the interactions of humans with virtual agents in spoken dialogue systems (Fujie et al, 2004;Ward & DeVault, 2016;Ward & Tsukahara, 1999) and in natural conversations, usually in cross-cultural or comparative settings (e.g., Cutrone, 2005Cutrone, , 2014Dingemanse & Liesenfeld, 2022;Li, 2006;Tottie, 1991;Xudong, 2008;Young & Lee, 2004).…”