Semi-autonomous avatar enabling unconstrained parallel conversations –seamless hybrid of WOZ and autonomous dialogue systems–

Abstract

Many people are now engaged in remote conversations for a wide variety of scenes such as interviewing, counseling, and consulting, but there is a limited number of skilled experts. We propose a novel framework of parallel conversations with semi-autonomous avatars, where one operator collaborates with several remote robots or agents simultaneously. The autonomous dialogue system mostly manages the conversation, but switches to the human operator when necessary. This framework circumvents the requirement for autonomous systems to be completely perfect. Instead, we need to detect dialogue breakdown or disengagement. We present a prototype of this framework for attentive listening.

Simultaneous Dialogue Services Using Multiple Semiautonomous Robots in Multiple Locations by a Single Operator: A Field Trial on Souvenir Recommendation

“…This is attributed to the operator's ability to tailor presentations to users rather than simply showcasing goods. These findings support the effectiveness of combining teleoperated and autonomous systems for spoken conversation robots, as proposed in previous work [7], [8]. The results also offer insights for designing systems that facilitate operators' interactions in such situations.…”

“…Zheng et al introduced a semiautonomous operation system where the operator adjusts intervention timing in navigation and conversation tasks simultaneously, guided by a proactive timing model [14], [15]. Kawahara et al proposed a framework for a semiautonomous teleoperation system wherein one operator engages in simultaneous conversations with multiple participants in a pseudo manner [7]. Kawai et al introduced a simultaneous job interview system, enabling one operator to conduct three interviews simultaneously [8].…”

“…In response to this demand, researchers have introduced semiautonomous teleoperation systems that empower a single operator to engage with multiple virtual agents concurrently, facilitating the simultaneous provision of spoken conversations [7], [8]. In these systems, conversations are primarily handled autonomously, with the operator monitoring the conversation history.…”