Robot Ego‐Noise Suppression with Labanotation‐Template Subtraction

Abstract

In this study, we aim to improve automatic-speech-recognition (ASR) accuracy in the presence of robot ego-noise toward a better human-robot interaction. Although several noise reduction methods have been proposed to increase ASR accuracy or signal-to-noise ratio (SNR) by predicting ego-noises through a short-time motion-template subtraction or a neural network, these methods showed poor performance in some practical use cases, such as attenuating long-term motion-associated ego-noise. Based on the motion-template subtraction method, we address the problem of creating ego-noise templates associated with a wide variety of robot motions. For representing robot motions, we employ a dance notation referred to as Labanotation. The rationales behind our approach are: (i) Labanotation allows quantizing infinite motion patterns using a finite number of Labanotation combinations; (ii) Labanotation-based motion description is hardware-independent; and (iii) long-time noise templates facilitate the localization of noise templates in a speech-with-noise signal compared to short-time templates. The effectiveness of the Labanotation-template subtraction (LTS) method was tested for five commercial ASRs in terms of ASR accuracy, SNR, and source-to-distortion ratio. We show that LTS leads to a reasonable performance, comparable to the other methods. The contribution of this study is (i) to propose to use Labanotation to reasonably collect noise templates, (ii) to demonstrate the practical effectiveness of LTS as well as examples of Labanotations for household actions.

A Learning-from-Observation Framework: One-Shot Robot Teaching for Grasp-Manipulation-Release Household Operations

“…As part of our efforts to develop a realistic robotic operation system, we have integrated the proposed system with a learning-from-observation system (Fig. 17) that includes a speech interface [44], [45], a visual teaching interface [33], a reusable library of robot actions [46], and a simulator for testing robot execution [47]. Please refer to the respective papers for the results of robot execution, as it is beyond the scope of this paper.…”

“…The user sends a query to the robot system via text or microphone input. Microphone input is noise-suppressed to prevent the robot's ego noise from interfering with recognition [7], [8]and then converted to text using a third-party text-to-speech technology [9]. The robot system then generates a prompt for the GPT-3/ChatGPT model based on this input.…”

ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application

“…As part of our efforts to develop a realistic robotic operation system, we have integrated our proposed task planner with a learning-from-observation system (Fig. 19) incorporating a speech interface [45], [46], a visual teaching interface [47], a reusable robot skill library [48], [49], and a simulator [50]. The code for the teaching system is available at: https://github.com/microsoft/ cohesion-based-robot-teaching-interface.…”

Effect of Face Masks on Automatic Speech Recognition Accuracy for Mandarin

“…Most prior research on the factors that affect ASR has focused on noise, speaker accent, speaker age, and multiple speakers. Noise persists as a significant hurdle to developing ASR, and many approaches have been proposed to enhance the robustness of ASR systems [18,19]. Our previous study suggested a method to associate the Articulation Index to estimate the influence of stationary noise on the ASR word accuracy (ACC) [20].…”