Multimodal Speaker Recognition in a Conversation Scenario

Abstract

As a step toward the design of a robot that can take part to a conversation we propose a robotic system that, taking advantage of multiple perceptual capabilities, actively follows a conversation among several human subjects. The essential idea of our proposal is that the robot system can dynamically change the focus of its attention according to visual or audio stimuli to track the actual speaker throughout the conversation and infer her identity.

No Need to Scream: Robust Sound-Based Speaker Localisation in Challenging Scenarios

“…Speaker Verification While traditionally this task has been addressed relying on Gaussian Mixture Models (e.g. [19,21]), recent advances in machine learning, particularly in the form of deep learning architectures (e.g. [8,9]) have dictated and driven the development of new methods able to achieve great precision, and to overcome the need of defining hand-crafted features.…”

“…Speaker Localisation Speaker or, more generally, sound source localisation, has followed a similar pattern, and more traditional geometrical methods [19,22] have been now superseded by deep learning approaches, such as [7,14]. Both those studies rely on cross-correlation information to train CNN-based models to perform localisation.…”

“…Literature in robotics has provided us with several speaker detection and recognition frameworks, most of which rely on face and voice characteristics (see [19,26] among others). Despite the accuracy of such frameworks, situations where noise might play a crucial role and heavily compromise the quality of the sound perceived by the robot have not been yet investigated.…”

Learning to Listen to Your Ego-(motion): Metric Motion Estimation from Auditory Signals

“…For long time, robot audition has mainly concerned the development of human-robot interaction frameworks (e.g. [6]). More recently, the robotics community has started investigating auditory perception in a wider perspective.…”

“…In audio-based classification, Mel-frequency cepstrum coefficients (MFCCs) [6] have been traditionally used as feature representations of the signals. However, recent studies proved that the performance of classification systems relying on MFCCs is greatly reduced in the presence of noise [7,19].…”