The complementary roles of auditory and motor information evaluated in a Bayesian perceptuo-motor model of speech perception.

Abstract

There is a consensus concerning the view that both auditory and motor representations intervene in the perceptual processing of speech units. However, the question of the functional role of each of these systems remains seldom addressed and poorly understood. We capitalized on the formal framework of Bayesian Programming to develop COSMO (Communicating Objects using Sensory-Motor Operations), an integrative model that allows principled comparisons of purely motor or purely auditory implementations of a speech perception task and tests the gain of efficiency provided by their Bayesian fusion. Here, we show 3 main results: (a) In a set of precisely defined “perfect conditions,” auditory and motor theories of speech perception are indistinguishable; (b) When a learning process that mimics speech development is introduced into COSMO, it departs from these perfect conditions. Then auditory recognition becomes more efficient than motor recognition in dealing with learned stimuli, while motor recognition is more efficient in adverse conditions. We interpret this result as a general “auditory-narrowband versus motor-wideband” property; and (c) Simulations of plosive-vowel syllable recognition reveal possible cues from motor recognition for the invariant specification of the place of plosive articulation in context that are lacking in the auditory pathway. This provides COSMO with a second property, where auditory cues would be more efficient for vowel decoding and motor cues for plosive articulation decoding. These simulations provide several predictions, which are in good agreement with experimental data and suggest that there is natural complementarity between auditory and motor processing within a perceptuo-motor theory of speech perception.

The role of production abilities in the perception of consonant category in infants

“…This view adequately construes the phonology-phonetic interface in a way compatible with neurocognitive data on speech communication (Laurent et al, 2017;Loevenbruck et al, 2018;Patri, Perrier, Schwartz, & Diard, 2018). The present data offer experimental evidence in favor of such a multisensory-motor scenario in the course of speech development.…”

“…The claim in the Perception-for-Action-Control Theory (Laurent et al, 2017;Schwartz et al, 2012) is that when infants begin to babble, at around 7 months, they discover the articulatory gestures and configurations associated with [bV], [dV], and [gV] syllables and they realize that these configurations belong to three natural articulatory-though not acoustic-classes. Hence, the proposal that phonetic classes are defined by mixed acoustic/auditory, somatosensory, and articulatory/motor properties that are learnt during the joint development of speech perception and production.…”

“…(adapted from Laurent, Barnaud,Schwartz, Bessière, & Diard, 2017, fig. 10), which provides the typ-ical representation of vowels in terms of acoustic F1-F2 formants (Figure 1, top) and of plosives in vowel (V) contexts, in terms of F2-F3 formants (Figure 1, bottom).…”

The motor system’s [modest] contribution to speech perception

“…This version claims the motor system influences speech perception, but it is not sufficient or essential for perception, suggesting instead that the motor system plays an ancillary role in which it is primarily engaged during noisy or degraded listening scenarios (Wilson, 2009;Wu et al, 2014). Additional support comes from a computational model of speech recognition in which auditory and motor theories were found to be indistinguishable during ideal listening conditions but not during degraded conditions (Barnaud, Bessière, Diard, & Schwartz, 2018;Laurent, Barnaud, Schwartz, Bessière, & Diard, 2017). Henceforth, we will focus on this weaker version of the motor theory which seems to be at the heart of most recent scientific literature on the subject.…”

The Motor System's [Modest] Contribution to Speech Perception

“…This version claims the motor system influences speech perception, but it is not sufficient or essential for perception, suggesting instead that the motor system plays an ancillary role in in which it is primarily engaged during noisy or degraded listening scenarios (Wilson, 2009;Wu et al, 2014). Additional support comes from a computational model of speech recognition in which auditory and motor theories were found to be indistinguishable during ideal listening conditions but not during degraded conditions MOTOR CONTRIBUTION TO SPEECH PERCEPTION 5 (Barnaud, Bessière, Diard, & Schwartz, 2018;Laurent, Barnaud, Schwartz, Bessière, & Diard, 2017). Henceforth, we will focus on this weaker version of the motor theory which seems to be at the heart of most recent scientific literature on the subject.…”