Sound Source Localization and Separation

Nakadai, Kazuhiro; Nakamura, Keisuke

doi:10.1002/047134608x.w8266

Cited by 9 publications

(3 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Within the field of robot audition, Sound Source Localisation (SSL) in robotic platforms has proven its importance [160][161][162] since these systems may allow a robot to improve its situational awareness and to complement other perceptual systems. Source localisation systems are currently applied in rescue environments in which visual contact is not available [163], in mapping tasks [164], or even in tasks focused on locating a human speaker in robot-assistant scenarios [165].…”

Section: Touch Gesture Localisation Systemmentioning

confidence: 99%

Affect Display Recognition Through Tactile and Visual Stimuli in a Social Robot

Marques-Villarroya¹,

Gamboa-Montero²,

Jumela-Yedra³

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This thesis is distributed under license "Creative Commons Atributtion -NonCommercial -Non Derivatives".A toda mi familia.En primer lugar tengo que agradecer por todo el trabajo a mis directores, Fernando y José Carlos, y a mi tutor, Miguel Ángel, por toda su ayuda. Sin duda alguna, de no ser por ellos, todo este trabajo no podría haber sido posible. A Fernando quiero agradecerle su cercanía, sus ideas y todo ese dinamismo tremendamente contagioso que ha permitido que este trabajo tuviera nuevas opciones cuando parecía que habíamos tocado hueso. Quiero agradecerle las charlas, los consejos y los paseos juntos divagando sobre cualquier tema que se nos viniese a la cabeza. A José Carlos quiero darle las gracias por todo su apoyo, serenidad y buen juicio, por estar ahí siempre que lo he necesitado, y, sobre todo, por estar ahí cuando no sabía que le necesitaba. Quiero agradecerle tanto por su paciencia conmigo y su sensatez, como por los buenos momentos y las risas juntos. A Miguel Ángel tengo que agradecerle toda su vasta experiencia, por los consejos en momentos críticos del trabajo y por su guía y buen criterio. Él ha sido el que me ha brindado esta enorme oportunidad y, sobre todo, quiero agradecerle por la estabilidad y la seguridad que siempre me ha dado. Por último, aunque no forman parte 'oficial' de este trabajo, tengo que dar las gracias a María y a Álvaro también, por todo su apoyo. Siempre han estado ahí cuando he necesitado una mano o simplemente charlar, y también son una parte importante de este trabajo.I would like to switch back to English in just this paragraph to thank the Intelligent Robots and Systems Group (IRSg) for all the help they gave me during my stay in Lisbon. It was an amazing experience for me I will always remember it. I want to thank Pedro Lima for the opportunity to spend my stay there, and, above all, I would like to thank Meysam Basiri for his support and mentorship. Muito obrigado! Ahora quiero hablar de mi familia. Sin ellos, definitivamente nada de lo que he realizado hasta ahora habría sido posible. Siempre han estado entre bambalinas para levantarme cuando me he caído y para abrazarme cuando más lo he necesitado. Quiero agradecer a mis padres, a Juanjo y a Begoña, por todo el amor, el apoyo y la guía que han volcado sobre mí durante todo este tiempo. Siempre me han dado lo mejor de sí mismos y no tengo palabras para describir todo lo que les agradezco lo que han hecho y siguen haciendo por mí. A Guille, mi hermano, quiero darle las gracias también por todo su cariño, su ayuda, las risas que siempre compartimos y, sin duda alguna, por aguantarme todo este tiempo. Quiero darles las gracias a mis abuelos, a Mariano, a Julia, a Antonio y a Mercedes, por no darme otra cosa que no fuese su amor, por todo el orgullo que siempre me han demostrado y por apoyarme incondicionalmente en cada decisión que he tomado hasta ahora. No me quiero olvidar de mis tíos y de mis primos, haciendo una mención especial a mi prima Paloma, por todo el cariño que siempre me han mostrado. Por último, qui...

show abstract

Section: Touch Gesture Localisation Systemmentioning

confidence: 99%

Affect Display Recognition Through Tactile and Visual Stimuli in a Social Robot

Marques-Villarroya¹,

Gamboa-Montero²,

Jumela-Yedra³

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Source Localization in Robotics Robotic systems use microphone arrays to perform sound source localization, often via signal processing techniques on the audio stream alone [51,60]. To focus on a sound, such as a human speaker, the microphone can be actively steered towards the localized source (e.g., [4,50,10]).…”

Section: Related Workmentioning

confidence: 99%

Move2Hear: Active Audio-Visual Source Separation

Majumder¹,

Al-Halah²,

Grauman³

2021

Preprint

View full text Add to dashboard Cite

We introduce the active audio-visual source separation problem, where an agent must move intelligently in order to better isolate the sounds coming from an object of interest in its environment. The agent hears multiple audio sources simultaneously (e.g., a person speaking down the hall in a noisy household) and must use its eyes and ears to automatically separate out the sounds originating from the target object within a limited time budget. Towards this goal, we introduce a reinforcement learning approach that trains movement policies controlling the agent's camera and microphone placement over time, guided by the improvement in predicted audio separation quality. We demonstrate our approach in scenarios motivated by both augmented reality (system is already co-located with the target object) and mobile robotics (agent begins arbitrarily far from the target object). Using state-of-the-art realistic audio-visual simulations in 3D environments, we demonstrate our model's ability to find minimal movement sequences with maximal payoff for audio source separation. Project: http://vision. cs.utexas.edu/projects/move2hear.

show abstract

“…The localization of sound sources based on an audio signal has been explored in robotics for both static [31,36] and dynamic sounds [17,22,34,45]. Early work in combining audio and vision focused on using Gaussian Processes [24] or canonical correlation analysis [29].…”

Section: Sound Source Localizationmentioning

confidence: 99%

Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Sounds

Younes¹,

Honerkamp²,

Welschehold³

et al. 2021

Preprint

View full text Add to dashboard Cite

Audio-visual navigation combines sight and hearing to navigate to a sound-emitting source in an unmapped environment. While recent approaches have demonstrated the benefits of audio input to detect and find the goal, they focus on clean and static sound sources and struggle to generalize to unheard sounds. In this work, we propose the novel dynamic audio-visual navigation benchmark which requires to catch a moving sound source in an environment with noisy and distracting sounds. We introduce a reinforcement learning approach that learns a robust navigation policy for these complex settings. To achieve this, we propose an architecture that fuses audio-visual information in the spatial feature space to learn correlations of geometric information inherent in both local maps and audio signals. We demonstrate that our approach consistently outperforms the current state-of-the-art by a large margin across all tasks of moving sounds, unheard sounds, and noisy environments, on two challenging 3D scanned real-world environments, namely Matterport3D and Replica. The benchmark is available at http://dav-nav.cs.uni-freiburg.de.

show abstract

Sound Source Localization and Separation

Cited by 9 publications

References 82 publications

Affect Display Recognition Through Tactile and Visual Stimuli in a Social Robot

Affect Display Recognition Through Tactile and Visual Stimuli in a Social Robot

Move2Hear: Active Audio-Visual Source Separation

Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Sounds

Contact Info

Product

Resources

About