A neural network model of the articulatory-acoustic forward mapping trained on recordings of articulatory parameters

Abstract

Three neural network models were trained on the forward mapping from articulatory positions to acoustic outputs for a single speaker of the Edinburgh multi-channel articulatory speech database. The model parameters (i.e., connection weights) were learned via the backpropagation of error signals generated by the difference between acoustic outputs of the models, and their acoustic targets. Efficacy of the trained models was assessed by subjecting the models' acoustic outputs to speech intelligibility tests. The results of these tests showed that enough phonetic information was captured by the models to support rates of word identification as high as 84%, approaching an identification rate of 92% for the actual target stimuli. These forward models could serve as one component of a data-driven articulatory synthesizer. The models also provide the first step toward building a model of spoken word acquisition and phonological development trained on real speech.

Hearing speech sounds: Top-down influences on the interface between audition and speech perception

“…Links between perception and production develop in infancy through babbling; a process that infants use to tune their production of speech to match the speech sounds that they perceive in their linguistic environment (Doupe and Kuhl, 1999;Kello and Plaut, 2004;Werker and Tees, 1999). These links play a central role in the development and maintenance of categorical representations of speech.…”

Vowel recognition from continuous articulatory movements for speaker-dependent applications

“…The need for this technology was discussed previously by Paush [1], who wanted to improve oral communication in patients with cerebral palsy by playing synthesized speech acoustics from articulatory movement directly, a goal that proved to be extremely challenging because human can produce the same sound in different ways of articulation. That is, the mapping between articulatory movements to speech is many-to-one [4], [5], [6].…”

Estimating the control parameters of an articulatory model from electromagnetic articulograph data

“…The method proposed in this paper may be regarded as an indirect way to map EMA information to speech acoustics: a problem that is otherwise addressed via data-driven statistical methods. 40,41 Our main motivation for this work has been the evaluation of analysis-by-synthesis speech inversion methods. 7,32 Direct evaluation of such methods in the articulatory space is difficult, since relevant data are rare.…”