Improving Multilingual Interaction for Consumer Robots through Signal Enhancement in Multichannel Speech

Abstract

Social robotics have become a trend in contemporary robotics research, since they can be successfully used in a wide range of applications. One of the most fundamental communication skills a consumer robot must have is the oral interaction with a human, in order to provide feedback or accept commands. There are quite a few well established Automatic Speech Recognition (ASR) tools, however without providing efficient results, especially in less popular languages, and more importantly under noisy conditions. The current paper investigates different voice activity detection and noise elimination methodologies to be used with ASRbased oral interaction with an affordable budget robot, NAO v4. Acoustically semi-stationary environments are assumed, which in conjunction to the high background noise of the NAO's microphones make the ASR quite difficult to succeed. Full Title:Improving multilingual interaction for consumer robots through signal enhancement in multichannel speechAdditional Information: Question ResponseHas this article previously been published at a conference or convention?No Please enter the text of an appropriate cover letter to accompany your submission, briefly describing the article you want to submit and saying why you believe it is suitable for publication in the AES Journal.Within the context of this paper several single and multi-channel VAD/noise estimation and noise elimination strategies were implemented and evaluated towards ASR in acoustically semi-stationary environments. This work aspires to be assistive in the consumer robotics domain, since the utilization of expensive hardware (microphones or robots in general) is usually out of the question, thus a robust VAD and noise reduction approach should exist. The main novelty of this paper does not lie in the implementations per se but in the adaptation approach followed, aiming to optimize the parameters of each method in order to improve ASR results. Concluding, we believe that the current work is eligible for submission in the AES Journal, since it researches a realistic problem current consumer robots have, from a signal processing point of view.Social robotics have become a trend in contemporary robotics research, since they can be successfully used in a wide range of applications. One of the most fundamental communication skills a consumer robot must have is the oral interaction with a human, in order to provide feedback or accept commands. There are quite a few well established Automatic Speech Recognition (ASR) tools, without however providing efficient results, especially in less popular languages, and more importantly under noisy conditions. The current paper investigates different voice activity detection and noise elimination methodologies to be used with ASRbased oral interaction with an affordable budget robot, NAO v4. Acoustically quasi-stationary environments are assumed, which in conjunction with the high background noise of the NAO's microphones make the ASR quite difficult to succeed. 8

Robust Audio Content Classification Using Hybrid-Based SMD and Entropy-Based VAD

“…The VAD differs from speech/music discrimination (SMD). VAD discriminates between noise and speech while SMD discriminates between speech and music [1][2][3][4][5][6][7].…”

“…Knowing how to build a management of multimedia information for AV (audio/video) indexing and retrieval is becoming extremely important. In the field of AV indexing and retrieval, the speech/music discrimination (SMD) is a very crucial task for the audio content classification (ACC) system or general audio detection and classification (GADC) [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18]. In recently, the SMD literatures have been presented in different application [19][20][21][22][23][24] and closely related to retrieval of audio content indexing [20].…”