Integration of sound source localization and separation to improve Dialogue Management on a robot

Fréchette, Maxime; Létourneau, Dominic; Valin, Jean‐Marc; Michaud, François

doi:10.1109/iros.2012.6385565

Cited by 11 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This Reliability-Weighted Phase Transform (RWPhaT) strategy results in a new adaptive frequency weight Ψ(f ). This GCC strategy is still used in [86], on a 8-microphone array embedded on the Spartacus robot, to show the efficiency of a complete artificial audition system for speech recognition and dialogue management. Different adaptations of the PhaT processor have also been proposed.…”

Section: Musicmentioning

confidence: 99%

A survey on sound source localization in robotics: From binaural to array processing methods

Argentieri

Danès

Souères

2015

Computer Speech & Language

133

View full text Add to dashboard Cite

This paper attempts to provide a state-of-the-art of sound source localization in Robotics. Noticeably, this context raises original constraints-e.g. embeddability, real time, broadband environments, noise and reverberation-which are seldom simultaneously taken into account in Acoustics or Signal Processing. A comprehensive review is proposed of recent robotics achievements, be they binaural or rooted in Array Processing techniques. The connections are highlighted with the underlying theory as well as with elements of physiology and neurology of human hearing.

show abstract

Section: Musicmentioning

confidence: 99%

A survey on sound source localization in robotics: From binaural to array processing methods

Argentieri

Danès

Souères

2015

Computer Speech & Language

133

View full text Add to dashboard Cite

show abstract

“…The following step in our research is to explore the use of SSL for improving the performance of ASR. Some interesting examples in this direction are presented in [35], [36], and [37]. These approaches make use of microphone arrays to localize the speech sources in the environment.…”

Section: B Computational Background and Related Workmentioning

confidence: 99%

Enhanced Robot Speech Recognition Using Biomimetic Binaural Sound Source Localization

Dávila-Chacón

Liu

Wermter

2019

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Inspired by the behavior of humans talking in noisy environments, we propose an embodied embedded cognition approach to improve automatic speech recognition (ASR) systems for robots in challenging environments, such as with ego noise, using binaural sound source localization (SSL). The approach is verified by measuring the impact of SSL with a humanoid robot head on the performance of an ASR system. More specifically, a robot orients itself toward the angle where the signal-to-noise ratio (SNR) of speech is maximized for one microphone before doing an ASR task. First, a spiking neural network inspired by the midbrain auditory system based on our previous work is applied to calculate the sound signal angle. Then, a feedforward neural network is used to handle high levels of ego noise and reverberation in the signal. Finally, the sound signal is fed into an ASR system. For ASR, we use a system developed by our group and compare its performance with and without the support from SSL. We test our SSL and ASR systems on two humanoid platforms with different structural and material properties. With our approach we halve the sentence error rate with respect to the common downmixing of both channels. Surprisingly, the ASR performance is more than two times better when the angle between the humanoid head and the sound source allows sound waves to be reflected most intensely from the pinna to the ear microphone, rather than when sound waves arrive perpendicularly to the membrane.

show abstract

“…Speech-based elements of teleoperation include speech understanding/synthesis (Marin, Vila, Sanz, & Marzal, 2002) and scripted speech acts wherein the humanoid can be controlled by issuing speech commands (Lu, Liu, Chen, & Huang, 2010). Additionally, speech recognition (supported by sound localization when environments are very noisy) can be used by a mobile robot to enable it to interact with bystanders/clients in the environment (Fréchette, Létourneau, Valin, & Michaud, 2012;Valin, Yamamoto, et al, 2007;Yamamoto et al, 2007).…”

Section: Sight and Visualizationmentioning

confidence: 99%

Teleoperation and Beyond for Assistive Humanoid Robots

Goodrich¹,

Crandall²,

Barakova³

2013

Reviews of Human Factors and Ergonomics

View full text Add to dashboard Cite

In this review, we explore how teleoperation could potentially be applied to the management of humanoid robots, with an emphasis on humanoid robots that are used in assistive roles, including clinical therapies. Since there are very few examples of the remote operation of a full humanoid, the review emphasizes technologies that are potentially relevant to teleoperation of humanoids. Teleoperation of humanoid robots faces many of the same practical challenges associated with (a) traditional teleoperation, including latency and telepresence, and (b) teleoperating manipulators and other robots with high degrees of freedom. Teleoperation systems for humanoid robots must also address unique challenges triggered by strong emotional and social responses to humanoids-responses of both the operator and any humans who may interact with the humanoid. These challenges trigger new opportunities for redefining teleoperation to include scripting, programming by demonstration, speech production, and Wizard of Oz interaction. The challenges also provide opportunities to specialize existing modes of interaction and unique opportunities to develop humanoid-specific forms of teleoperation, such as controlling humanoids via exoskeleton-based or inertial sensors. We include in this review a survey of enabling technologies, a taxonomy of practical uses, and a list of emerging themes and approaches to teleoperating humanoid robots.

show abstract

Integration of sound source localization and separation to improve Dialogue Management on a robot

Cited by 11 publications

References 12 publications

A survey on sound source localization in robotics: From binaural to array processing methods

A survey on sound source localization in robotics: From binaural to array processing methods

Enhanced Robot Speech Recognition Using Biomimetic Binaural Sound Source Localization

Teleoperation and Beyond for Assistive Humanoid Robots

Contact Info

Product

Resources

About