Brain responses in humans reveal ideal observer-like sensitivity to complex acoustic patterns

Abstract

We use behavioral methods, magnetoencephalography, and functional MRI to investigate how human listeners discover temporal patterns and statistical regularities in complex sound sequences. Sensitivity to patterns is fundamental to sensory processing, in particular in the auditory system, because most auditory signals only have meaning as successions over time. Previous evidence suggests that the brain is tuned to the statistics of sensory stimulation. However, the process through which this arises has been elusive. We demonstrate that listeners are remarkably sensitive to the emergence of complex patterns within rapidly evolving sound sequences, performing on par with an ideal observer model. Brain responses reveal online processes of evidence accumulationdynamic changes in tonic activity precisely correlate with the expected precision or predictability of ongoing auditory input-both in terms of deterministic (first-order) structure and the entropy of random sequences. Source analysis demonstrates an interaction between primary auditory cortex, hippocampus, and inferior frontal gyrus in the process of discovering the regularity within the ongoing sound sequence. The results are consistent with precision based predictive coding accounts of perceptual inference and provide compelling neurophysiological evidence of the brain's capacity to encode high-order temporal structure in sensory signals.A ccumulating work suggests that the brain is sensitive to statistical regularities in sensory input, at multiple time scales (1-9). The auditory system has been a useful testbed to investigate these processes (2, 3, 9-13), largely due to the vantage point provided by the mismatch negativity (MMN) paradigm (12, 13). The MMN is an auditory-evoked response generated by sounds violating some regular aspect of the prior sequence and is hypothesized to reflect a discrepancy between the memory trace, or expectations, generated by the standard stimulus, and the deviant information (12,13). A large body of MMN work has demonstrated that listeners are sensitive to the violation of a variety of acoustic sequences, including very complex regularities (14, 15), and interpreted as indirect evidence for exquisite sensitivity to patterns in sound.Due to the physical constraints that characterize animate objects in the environment, sounds emanating from those sources are usually statistically regular and often repetitive (e.g., flapping wings and locomotion sounds). The ability to discover regularities within the sensory input is therefore a critical aspect of scene analysis: providing the anchor that enables an observer to identify and track a behaviorally relevant signal from within the brouhaha of a busy scene. Detecting temporally recurrent auditory features enables listeners to recognize auditory objects (because most auditory signals only have meaning as patterns over time), but also to form rules, or models, that characterize the past and expected behavior of objects within the environment (4, 16). Indeed, experimental work demonstrat...

Detecting and representing predictable structure during auditory scene analysis

“…One proposal is that attention (like regularity) acts to determine the inferred precision of sensory input (Friston, 2009; Barascud et al, 2016). In this view, attention increases precision (and neural responses) when sensory signals are task-relevant.…”

Is predictability salient? A study of attentional capture by auditory patterns

“…Recently, Barascud et al . [19] provided direct evidence of the process of regularity extraction in more complex tone sequences. They used rapid sequences of tones with frequencies changing in a regular, cyclical pattern, and matched sequences of tones arranged in a random order.…”

Computational Models of Auditory Scene Analysis: A Review

“…That is, the sustained response increases with increasing predictability, suggesting that this representation is precision weighted as opposed to being a result of adaptation or a correlate of prediction error, both of which would suggest a response decrement (Mathys et al, 2011; Teki et al, 2016). Offset responses to auditory objects (patterned sequences) are compatible with the notion of predictive auditory object representations (Andreou et al, 2015; Barascud et al, 2016), with some caveats; offsets responses to repeating pitch patterns require about three violations (Barascud et al, 2016), and offset responses to repeating temporal patterns are only elicited by isochronous sequences, whereas offset responses to non-isochronous sequences seem to require attention (Andreou et al, 2015). In summary, while neural signatures have been found for most of the important perceptual effects of ASA, exactly how separable object representations are instantiated in cortical networks is still largely unknown.…”

“…Tone clouds have been used in information masking designs measuring the effect of auditory stream segregation on detecting tone repetition within a protected frequency range of the cloud (e.g., Kidd et al, 1994; Elhilali et al, 2009b; Akram et al, 2014a). They also allow the creation of target patterns for detection (e.g., Kumar et al, 2014; Barascud et al, 2016) as well as variable backgrounds within which repeating target patterns can be detected. For the latter purpose, Teki et al (2011) created a sound configuration consisting of a series of tonal complexes composed of random frequencies, which were presented without pause (Figure 2).…”

“…In support of this idea, cortical responses to a target speaker have been shown to be invariant with respect to irrelevant object details (such as intensity changes) (Ding and Simon, 2014; Simon, 2015). By studying transitions between random and patterned tone sequences it has been shown that steady-state responses in auditory cortex, inferior frontal gyrus, and hippocampus correlate with the predictability of the sequence (Barascud et al, 2016; Kumar et al, 2016). That is, the sustained response increases with increasing predictability, suggesting that this representation is precision weighted as opposed to being a result of adaptation or a correlate of prediction error, both of which would suggest a response decrement (Mathys et al, 2011; Teki et al, 2016).…”