Detecting Robot-Directed Speech by Situated Understanding in Physical Interaction

Abstract

keywords: robot-directed speech detection, multimodal semantic confidence, human-robot interaction SummaryIn this paper, we propose a novel method for a robot to detect robot-directed speech: to distinguish speech that users speak to a robot from speech that users speak to other people or to themselves. The originality of this work is the introduction of a multimodal semantic confidence (MSC) measure, which is used for domain classification of input speech based on the decision on whether the speech can be interpreted as a feasible action under the current physical situation in an object manipulation task. This measure is calculated by integrating speech, object, and motion confidence with weightings that are optimized by logistic regression. Then we integrate this measure with gaze tracking and conduct experiments under conditions of natural human-robot interactions. Experimental results show that the proposed method achieves a high performance of 94% and 96% in average recall and precision rates, respectively, for robot-directed speech detection. * 1 Commands made in Japanese have been translated into English in this paper.

Dialogue with Robots to Support Symbiotic Autonomy

“…In the Robotic field, speech-based approaches have been applied to deploy robotic platforms in a wide plethora of environments. These techniques have been used in manipulation tasks [4], and for wheeled platforms [5,6].Dialogue has also been employed to instruct robots to accomplish a given unknown task, such as giving a tour [7], delivering objects [8], or manipulating them [9]. Other related works have combined speech-based approaches with other types of interactions [6,10].…”

Discriminating Unknown Objects from Known Objects Using Image and Speech Information

“…Gaussian models were learned using these features by MAP adaptation. The confidence of the objects is written as follows [13]:…”

“…The speech recognition confidence measure and object recognition confidence measure are integrated by the following logistic regression function [13]:…”

Selection of Unknown Objects Specified by Speech Using Models Constructed from Web Images

“…The confidences of speech and image recognition results are integrated by the following logistic regression function [13], and the integrated confidence is used for object recognition and selection.…”