Automatic classification of question turns in spontaneous speech using lexical and prosodic evidence

Abstract

The ability to identify speech acts reliably is desirable in any spoken language system that interacts with humans. Minimally, such a system should be capable of distinguishing between questionbearing turns and other types of utterances. However, this is a non-trivial task, since spontaneous speech tends to have incomplete syntactic, and even ungrammatical, structure and is characterized by disfluencies, repairs and other non-linguistic vocalizations that make simple rule based pattern learning difficult. In this paper, we present a system for identifying question-bearing turns in spontaneous multi-party speech (ICSI Meeting Corpus) using lexical and prosodic evidence. On a balanced test set, our system achieves an accuracy of 71.9% for the binary question vs. non-question classification task. Further, we investigate the robustness of our proposed technique to uncertainty in the lexical feature stream (e.g. caused by speech recognition errors). Our experiments indicate that classification accuracy of the proposed method is robust to errors in the text stream, dropping only about 0.8% for every 10% increase in word error rate (WER).

Measuring the voice.

“…In addition to directly impacting the meaning of some utterances, such as "okay" (Gravano, Benus, Hirschberg, German, & Ward, 2008), prosody is used to structure spoken material (Grosz & Sidner, 1986;Nakajima & Allen, 1993), affecting the intended context of interpretation. Analysis of higher level prosody has also been shown to be useful in improving performance in many spoken language processing tasks, such as ASR (Zhang, Hasegawa-Johnson, & Levinson, 2004), speech synthesis (Demenko, Grocholewski, Wagner, & Szymanski, 2006), speaker identification (Weber, Manganaro, Peskin, & Shriberg, 2002), language (Tong, Ma, Zhu, Li, & Chang, 2006) and dialect identification (Rouas, 2007), story/topic segmentation (Rosenberg, Sharifi, & Hirschberg, 2007;, sentence segmentation (Shriberg, Stolcke, Hakkani-Tur, & Tur, 2000), discourse segmentation (Hirschberg & Nakatani, 1996;Nakatani, Hirschberg, & Grosz, 1995), extractive speech summarization (Maskey, Rosenberg, & Hirschberg, 2008), punctuation insertion (Christensen, Gotoh, & Renals, 2001), and speech act classification (Ananthakrishnan, Ghosh, & Narayanan, 2008;Shriberg et al, 1998).…”