Learning hierarchical sequence representations across human cortex and hippocampus

Abstract

Sensory input arrives in continuous sequences that humans experience as units, e.g., words and events. The brain’s ability to discover extrinsic regularities is called statistical learning. Structure can be represented at multiple levels, including transitional probabilities, ordinal position, and identity of units. To investigate sequence encoding in cortex and hippocampus, we recorded from intracranial electrodes in human subjects as they were exposed to auditory and visual sequences containing temporal regularities. We find neural tracking of regularities within minutes, with characteristic profiles across brain areas. Early processing tracked lower-level features (e.g., syllables) and learned units (e.g., words); while later processing tracked only learned units. Learning rapidly shaped neural representations, with a gradient of complexity from early brain areas encoding transitional probability, to associative regions and hippocampus encoding ordinal position and identity of units. These findings indicate the existence of multiple, parallel computational systems for sequence learning across hierarchically organized cortico-hippocampal circuits.

Cortical encoding of melodic expectations in human temporal cortex

“…Musical perception is strongly influenced by expectations (Bar et al 2006;Huron 2006;Kok et al 2012;Pearce 2018;Henin et al 2019). Violation of these expectations elicits distinct neural signatures that may underlie the emotional and intellectual engagement with music .…”

Pupil diameter tracks statistical structure in the environment

“…One possible source of pupil diameter modulation is extrastriate visual cortex, where the same paradigm we used here affects neural processing of facial information [21] and where lesions affect pupil diameter modulation by other visual features like color [22]. Statistical structure also entrains brain areas beyond visual cortex, including motor and premotor cortex [23]. Here, stimulation, e.g., of the frontal eye fields, induces pupil size changes in macaque monkeys [24, 25].…”

Network architectures supporting learnability

“…[115, p. 1072] While the notion that mind and object of thought are the same might initially appear fanciful and rather arcane, there are many metaphors and research programmes that reflect the human intuition that there exists some structural similarity in how we think about knowledge architecture and brain architecture. Moreover, emerging evidence offers preliminary support for one candidate operationalization of precisely the notion that mind and object of thought are intimately connected [39,40,[116][117][118]. When a mind is shown relational knowledge with a specified network architecture, brain activity reflects that architecture in a particular manner.…”

“…Such representations may depend upon the nature of the relations or the content being related, and may manifest distinctly in the scale accessible to fMRI compared to the scale accessible to cellular imaging. Finally, such representations may also differ across regions [118], being precise reflections of the graph or more akin to a predicate logic.…”

“…Is the given brain area (or the entire brain) a modeller, representer, processor, encoder, or all of the above? First, note that to the degree that the brain represents knowledge, correspondence between the network structure of neural representations and the network structure of object relations is perhaps expected based on recent empirical studies [39,40,[116][117][118] (although note that further studies are needed that explicitly compare the neural representations developed in response to different network structures). Second, to the degree that the brain processes information, correspondence between the network structure of informational connections and the network structure of the information is also perhaps expected [65,121].…”