Trial-by-trial predictions of subjective time from human brain activity

Abstract

Author ContributionsWR conceived of the study. MTS and WR designed and pre-registered the experiments and analyses. MTS collected, analyzed, and constructed models of human behavioral and neuroimaging data. ZF constructed the artificial network model and analyzed the data. MTS and WR wrote the manuscript. AKS and ZF provided critical revisions on the manuscript. AbstractHuman experience of time exhibits systematic, context-dependent deviations from objective clock time. For example, time is experienced differently at work than on holiday. The cognitive and neural bases of how time perception interacts with the content of experience remain unclear, and leading explanations of human time perception are not equipped to explain this interaction. We propose an alternative account of human time perception, based on the dynamics of sensory processing. Our approach naturally links content of experience with time perception through a common foundation in basic sensory processing. We provide evidence for this proposal in model-based analyses of the dynamics of perceptual processing in an artificial neural network and in the activity of human sensory cortex. Healthy human participants watched naturalistic, silent videos and estimated their duration while fMRI was acquired. The same videos were used as stimuli for the artificial network.We constructed a computational model that predicted video durations from salient events in the activity of the artificial network, or in participants' visual cortex. The artificial network model reproduced human-like duration estimates, including biases in estimation depending on the content of a given video. Most importantly, the model based on human visual cortex activity reproduced trial-by-trial biases in our human participants' subjective reports, whereas control models trained on auditory or somatosensory activity did not. Together, our results reveal that human subjective time is based on information arising during the processing of our dynamic sensory environment, providing a computational basis for an end-to-end account of time perception. Significance StatementOur perception of time depends on the contents of experience, reflected in expressions such as "a watched pot never boils". Prevailing accounts of human time perception cannot provide good explanations for this. We tested a new explanation: time perception arises from the processing of our dynamic sensory environment. Supporting this theory, human participants' duration reports of silent videos correlated with estimates we reconstructed from activity in the visual cortex of their brain while they watched those videos. This was not a brain-wide phenomenon; reconstructions based on other brain regions did not correlate with subjective reports of duration. Our results validate our new theory, linking content of experience and perception of time through a common foundation in basic sensory processing.

A predictive processing model of episodic memory and time perception

“…This pattern of results was reflected in a 2x2x3 ANOVA, with task, cognitive load, and scene type as factors, which revealed a significant main effect of scene (F (2, 6965) = 53.07, p < 0.001). This finding replicates previous, similar results indicating the difference in duration estimation for these naturalistic stimulus classes [4,7,24], but now also extends evidence for this effect to the retrospective task. Of greater interest is that the nature of the task by load interaction appears to qualitatively change with scene type.…”

“…For example, neural dynamics related to event boundaries in basic stimulus properties such as motion are detectable in lower level sensory areas like human MT 55,56,71 while higher complexity events will be detectable in the dynamics of higher cortical regions 56 . A context-related cortical specificity for salient events is also consistent with the finding that only the accumulation of salient events within the relevant sensory areas (ventral visual areas when watching silent videos) can reproduce human reports of subjective duration 13 . While structurally equipped to produce many properties of episodic memory, our present model was developed explicitly to address human time perception.…”

“…In support of the proposal that time can be estimated on the basis of activity in perceptual classification networks, a recent study [4] demonstrated that by simply tracking salient events in the activity of a CNN while it processed videos of natural scenes (from 1-64 seconds in duration) it was possible to generate a basis for prospective estimates of duration. In that study, model-based duration estimates replicated key features of human estimates for the exact same videos, including biases related to scene type (busy city scenes estimated as longer than quiet office scenes, for example; see also [7,24]). This model displayed conceptual similarities with the predictive processing account of perception wherein perception is proposed to operate as a function of both sensory predictions and current sensory stimulation, with perceptual content understood as the brain's "best guess" (Bayesian posterior) of the causes of current sensory input given the prior expectations or predictions [25][26][27][28][29].…”

Predictive processing as a systematic basis for identifying the neural correlates of consciousness

“…As a final example, the sense of the present as temporally extended may be explained by PP-like internal simulation (Grush, 2006); the sense of temporal flow of conscious experience -how one moment passes seamlessly into the next even when nothing much happens in the environment -may be explained by the dynamics of hierarchical Bayes that encode expected change in the environment (Hohwy et al, 2016;Sherman et al, 2020;Vogel et al, 2020Vogel et al, , 2019; this process may in turn relate to the alternation of percepts in binocular rivalry, as discussed above.…”

Predictive processing as a systematic basis for identifying the neural correlates of consciousness

“…As a final example, the sense of the present as temporally extended may be explained by PP-like internal simulation (Grush 2006); the sense of temporal flow of conscious experience -how one moment passes seamlessly into the next even when nothing much happens in the environment -may be explained by the dynamics of hierarchical Bayes that encode expected change in the environment (Hohwy, Paton et al 2016, Vogel, Falter-Wagner et al 2018, Vogel, Beeker et al 2019, Sherman, Fountas et al 2020; this process may in turn relate to the alternation of percepts in binocular rivalry, as discussed above.…”