Predictive coding with neural transmission delays: a real-time temporal alignment hypothesis

Abstract

Hierarchical predictive coding is an influential model of cortical organization, in which sequential hierarchical layers are connected by feedback connections carrying predictions, as well as feedforward connections carrying prediction errors. To date, however, predictive coding models have neglected to take into account that neural transmission itself takes time. For a time-varying stimulus, such as a moving object, this means that feedback predictions become misaligned with new sensory input. We present an extended model implementing both feed-forward and feedback extrapolation mechanisms that realigns feedback predictions to minimize prediction error. This realignment has the consequence that neural representations across all hierarchical stages become aligned in real-time. Using visual motion as an example, we show that the model is neurally plausible, that it is consistent with evidence of extrapolation mechanisms throughout the visual hierarchy, that it predicts several known motionposition illusions, and that it provides a solution to the temporal binding problem.

Predictive coding of visual motion in both monocular and binocular human visual processing

“…In a toy example, Area 1 sends visual information about the position of a moving object to Area 2, which in turn sends a ''prediction'' back to Area 1. That prediction is compared with input in Area 1 and any mismatch error is recursively minimized by adjusting the feedback signal to Area 1 to line up with its input at the time the signal arrives there (for details, see Hogendoorn & Burkitt, 2018b). Minimizing error therefore requires compensating for the delays incurred in both feed-forward and feed-back signaling.…”

When predictions fail: Correction for extrapolation in the flash-grab effect

“…In support of this, a range of physiological studies have revealed extrapolation mechanisms at various stages of the visual hierarchy, including the retina (Berry, Brivanlou, Jordan, & Meister, 1999), lateral geniculate nucleus (Sillito, Jones, Gerstein, & West, 1994), V1 (Jancke, Erlhagen, Schöner, & Dinse, 2004), V4 (Sundberg, Fallah, & Reynolds, 2006), MT (Maus, Fischer, & Whitney, 2013), and in both monocular and binocular populations (van Heusden, Harris, Garrido, & Hogendoorn, 2019). Using an electroencephalogram decoding approach, we recently showed that early cortical position signals are pre-activated ahead of predictably moving stimuli (Hogendoorn & Burkitt, 2018a), and argued that within the framework of hierarchical predictive coding (Rao & Ballard, 1999), such extrapolation mechanisms would be ubiquitous in the visual hierarchy (Hogendoorn & Burkitt, 2018b). We have previously argued that EEG correlates of the flash-grab effect are detectable so rapidly after presentation (;80 ms; Hogendoorn, Verstraten, & Cavanagh, 2015) that motion after the target could impossibly be processed on time to affect the (initial development of the) illusion.…”

Decoding Predictions and Violations of Object Position and Category in Time-resolved EEG

“…Overall, 422 our results were largely contrary to this idea. The remainder of this paper will discuss 423 the consequence of our results for each hypothesis and propose an explanation of the 424 findings in terms of generalised predictive coding (Friston & Kiebel, 2009;Friston, 425 Stephan, Li & Daunizeau, 2010) and the temporal realignment hypothesis 426 (Hogendoorn & Burkitt, 2019). 427…”