Optimizing endpointing thresholds using dialogue features in a spoken dialogue system

Abstract

This paper describes a novel algorithm to dynamically set endpointing thresholds based on a rich set of dialogue features to detect the end of user utterances in a dialogue system. By analyzing the relationship between silences in user's speech to a spoken dialogue system and a wide range of automatically extracted features from discourse, semantics, prosody, timing and speaker characteristics, we found that all features correlate with pause duration and with whether a silence indicates the end of the turn, with semantics and timing being the most informative. Based on these features, the proposed method reduces latency by up to 24% over a fixed threshold baseline. Offline evaluation results were confirmed by implementing the proposed algorithm in the Let's Go system.

Incremental dialogue processing in a micro-domain

“…In other words, the system bases its turn-taking decisions on a combination of ASR, prosody and silencethresholds, where the length of the threshold differs for different prosodic signals, and where reactions are planned already during the silence. (This is in contrast to Raux & Eskenazi (2008), where context-dependent thresholds are used as well, but only simple end-pointing is performed. )…”

“…Speakers appear to use other knowledge sources, such as prosody, syntax and semantics to detect or even project the end of the utterance. Attempts have been made to incorporate such knowledge sources for turn-taking decisions in spoken dialogue systems (e.g., Ferrer et al, 2002;Raux & Eskenazi, 2008). To do so, incremental dialogue processing is clearly needed.…”

Opportunities and Obligations to Take Turns in Collaborative Multi-Party Human-Robot Interaction

“…While this model simplifies processing, it fails to account for many aspects of human-human interaction such as hesitations, turn-taking with very short gaps or brief overlaps and backchannels in the middle of utterances (Heldner & Edlund, 2010). More advanced models for turn-taking have been presented, where the system interprets syntactic and prosodic cues to make continuous decisions on when to take the turn or give feedback, resulting in both faster response time and less interruptions (Raux & Eskenazi, 2008;Skantze & Schlangen, 2009;Meena et al, 2014).…”

An easy method to make dialogue systems incremental

“…The end of a request is determined by a condition EndT urnCond. It can be a long enough silence (Raux and Eskenazi, 2008;Wlodarczak and Wagner, 2013) in the case of vocal services or a carriage return for text systems. A dialogue turn is the time interval during which the user sends a request to the system and gets a response.…”