Asynchrony adaptation reveals neural population code for audio-visual timing

Abstract

The relative timing of auditory and visual stimuli is a critical cue for determining whether sensory signals relate to a common source and for making inferences about causality. However, the way in which the brain represents temporal relationships remains poorly understood. Recent studies indicate that our perception of multisensory timing is flexible—adaptation to a regular inter-modal delay alters the point at which subsequent stimuli are judged to be simultaneous. Here, we measure the effect of audio-visual asynchrony adaptation on the perception of a wide range of sub-second temporal relationships. We find distinctive patterns of induced biases that are inconsistent with the previous explanations based on changes in perceptual latency. Instead, our results can be well accounted for by a neural population coding model in which: (i) relative audio-visual timing is represented by the distributed activity across a relatively small number of neurons tuned to different delays; (ii) the algorithm for reading out this population code is efficient, but subject to biases owing to under-sampling; and (iii) the effect of adaptation is to modify neuronal response gain. These results suggest that multisensory timing information is represented by a dedicated population code and that shifts in perceived simultaneity following asynchrony adaptation arise from analogous neural processes to well-known perceptual after-effects.

Asymmetries in visuomotor recalibration of time perception: Does causal binding distort the window of integration?

“…Recalibration could be a linear shift with respect to baseline (A), a result that would be compatible with the explanation that recalibration involves a general update of a differential delay, without asymmetries due to intentional (causal) binding. Alternatively, there could be local non-linear biases in recalibrated time perception (B), similar to those that Roach et al (2011) report for audio-visual recalibration. Even though this kind of recalibration is non-linear, it is still symmetrical along the negative diagonal, which would indicate that the mechanisms of recalibration are not sensitive to direction (i.e., which modality leads the temporal order in the trained discrepancy).…”

“…Humans can recalibrate the perceived timing of multisensory events to compensate for the presence of small temporal discrepancies between the senses for a number of modality pairs, such as vision and audition or vision and touch (e.g., Fujisaki et al, 2004;Keetels & Vroomen, 2008;Di Luca et al, 2009;Roach et al, 2011;Yarrow et al, 2011). The perceived temporal order of a voluntary movement (e.g., a button press) and a sensory stimulus (e.g., a visual flash) is no exception from this (Stetson et al, 2006;Heron et al, 2009;Sugano et al, 2010;Sugano & Vroomen, 2012;Rohde & Ernst, 2013).…”

“…However, recent studies (e.g., Roach et al, 2011;Yarrow et al, 2011) have shown that an exclusive focus on PSS in temporal recalibration may not capture all aspects of temporal recalibration. Roach et al (2011) have used an interval estimation (IE) task to study audiovisual temporal recalibration.…”

“…Roach et al (2011) have used an interval estimation (IE) task to study audiovisual temporal recalibration. Participants had to estimate the length of intervals between a visual and an auditory stimulus.…”

Sensorimotor temporal recalibration within and across limbs.

“…Of course there may be data that our model cannot explain, and it may ultimately need to be expanded (for example by incorporating keying errors; GarciaPerez & Alcala-Quintana, 2012) or indeed supplanted entirely by a different type of model (e.g. Roach et al, 2011). At present, however, we consider our model to be as good a candidate as any.…”