A unified approach to speech production and recognition based on articulatory motor representations

Abstract

Abstract-We present a unified approach for speech production and recognition based on articulatory motor representations. The approach is inspired by the Motor theory and the discovery of Mirror neurons, and use motor representations for both reproduction and recognition of speech. A model of the vocal tract is used to create sound and the created sound is then mapped back to the motor representation using a neural network. To learn the map we mimic the behavior of a child that uses a combination of babbling and interaction with its caregiver to learn how to speak. Several different phases of babbling and interaction are identified and described. These help to overcome the inversion problem. The approach has been implemented on a humanoid robot, which has successfully learned to pronounce Swedish and Portuguese vowels. We have also studied how the different phases of babbling and interaction effect the error of the map and the achieved recognition rate when presented with vowels from different subjects. Finally we compare the recognition rates obtained using motor space with recognition rates obtained by directly using the acoustic parameters.

Multimodal Language Acquisition Based on Motor Learning and Interaction

“…The described architecture is an extension of the work in described in [23]. It includes a speech production module, a sensing unit, a sensorimotor coordination module and a memory unit, as shown in 1.…”

“…We then discuss the learning architecture and necessary resources that has to be implemented in a robot for it to be able to acquire early language structures through the described interactions. The architecture described is an extended version of the architecture presented in [23]. A similar architecture can also be found in [28], and some earlier work can be found in the DIVA model [19].…”