Bringing the Nonlinearity of the Movement System to Gestural Theories of Language Use: Multifractal Structure of Spoken English Supports the Compensation for Coarticulation in Human Speech Perception

Abstract

Coarticulation is the tendency for speech vocalization and articulation even at the phonemic level to change with context, and compensation for coarticulation (CfC) reflects the striking human ability to perceive phonemic stability despite this variability. A current controversy centers on whether CfC depends on contrast between formants of a speech-signal spectrogram—specifically, contrast between offset formants concluding context stimuli and onset formants opening the target sound—or on speech-sound variability specific to the coordinative movement of speech articulators (e.g., vocal folds, postural muscles, lips, tongues). This manuscript aims to encode that coordinative-movement context in terms of speech-signal multifractal structure and to determine whether speech's multifractal structure might explain the crucial gestural support for any proposed spectral contrast. We asked human participants to categorize individual target stimuli drawn from an 11-step [ga]-to-[da] continuum as either phonemes “GA” or “DA.” Three groups each heard a specific-type context stimulus preceding target stimuli: either real-speech [al] or [a], sine-wave tones at the third-formant offset frequency of either [al] or [aɹ], and either simulated-speech contexts [al] or [aɹ]. Here, simulating speech contexts involved randomizing the sequence of relatively homogeneous pitch periods within vowel-sound [a] of each [al] and [aɹ]. Crucially, simulated-speech contexts had the same offset and extremely similar vowel formants as and, to additional naïve participants, sounded identical to real-speech contexts. However, randomization distorted original speech-context multifractality, and effects of spectral contrast following speech only appeared after regression modeling of trial-by-trial “GA” judgments controlled for context-stimulus multifractality. Furthermore, simulated-speech contexts elicited faster responses (like tone contexts do) and weakened known biases in CfC, suggesting that spectral contrast depends on the nonlinear interactions across multiple scales that articulatory gestures express through the speech signal. Traditional mouse-tracking behaviors measured as participants moved their computer-mouse cursor to register their “GA”-or-“DA” decisions with mouse-clicks suggest that listening to speech leads the movement system to resonate with the multifractality of context stimuli. We interpret these results as shedding light on a new multifractal terrain upon which to found a better understanding in which movement systems play an important role in shaping how speech perception makes use of acoustic information.

Complexity in psychological self-ratings: implications for research and practice

“…Longrange temporal correlations can be understood as slow "waves" that are reflected by the time series [45,76]. The present results align with the ever growing empirical record of studies that have evidenced the presence and characteristics of long-range correlations and fractal scaling (i.e., a specific pattern of long-range correlations associated with selforganization [42]) in time and trial series of human physiology and performance [40][41][42][43][44][77][78][79][80][81][82][83][84][85][86]. Moreover, the temporal correlations changed over time, indicating multifractality, which is considered a strong indicator for multiplicative interactions across time scales as data-generating process [45].…”

Multifractal Mice: Measuring Task Engagement and Readiness-to-hand via Hand Movement

“…Multifractality is a measure of system complexity, developed in fluid mechanics, but now important in a range of fields [47], including physiology [6,34,55,75,83] and cognitive science [32,44,52,76]. Multifractality indicates the role of complex nonlinear interactions in a system's behaviour [42,52], and a large literature now demonstrates its presence in a wide range of human behaviours [15,16,23,33,35,46,57,65,76,77,84], often associated with adaptive flexibility [4,56,67].…”

Multifractality distinguishes reactive from proactive cascades in postural control

“…Multifractal nonlinearity moderates all influences of task constraints and intentional control on cascade dynamics. Indeed, extensive research has shown that multifractal nonlinearity of upright posture changes in response to task constraints [57] and intentions [58], and changes in multifractal nonlinearity influences participants’ perceptual judgments [53,59,60]. In the present case, we hypothesize that multifractal nonlinearity would shape all features articulated above: the scale-dependent cascade carrying the reactive response to destabilization, the scale-invariant cascade carrying proactive stabilization specifically under destabilizing task constraints, and the reconfiguration of cascade dynamics supporting intentional resetting of equilibrium points in general (Hypothesis-4).…”