Visuomotor Mismatch Responses as a Hallmark of Explaining Away in Causal Inference

Abstract

How are visuomotor mismatch responses in primary visual cortex embedded into cortical processing? We here show that mismatch responses can be understood as the result of a cooperation of motor and visual areas to jointly explain optic flow. This cooperation requires that optic flow is not explained redundantly by both areas, meaning that optic flow inputs to V1 that are predictable from motor neurons should be canceled (i.e., explained away). As a result, neurons in V1 represent only external causes of optic flow, which could allow the animal to easily detect movements that are independent of its own locomotion. We implement the proposed model in a spiking neural network, where coding errors are computed in dendrites and synaptic weights are learned with voltage-dependent plasticity rules. We find that both positive and negative mismatch responses arise, providing an alternative to the prevailing idea that visuomotor mismatch responses are linked to dedicated neurons for error computation. These results also provide a new perspective on several other recent observations of cross-modal neural interactions in cortex.

Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

“…by Pearson et al, 2021. Another type of multimodal generative model is learning of motor-to-visual forward models based on corollary discharges, wherein motor commands are taken to constitute a non-sensory modality. On a neural level, such models have been proposed by Hertäg and Sprekeler, 2020;Mikulasch et al, 2022, in contrast to our model without the ability to account for externally-caused optic flow. Interestingly, Hertäg and Sprekeler, 2020 demonstrated that signed prediction error responses emerge under visual feedback contingent with motor state in a biologically detailed model, and investigate the role of the involved interneurons.…”

“…In contrast to Mikulasch et al, 2022 andHertäg andSprekeler, 2020, the used datasets go beyond one-dimensional visual inputs and the model is capable of solving visual tasks such as figure-ground segmentation and, to a limited degree, classification. To our knowledge, the model put forward here is the first to integrate motor-to-sensory forward models and sensory-sensory predictive coding within a functional model of generative visual perception.…”

“…This endows the brain with neural circuits for mismatch (or prediction error)-based segmentation. While models demonstrating sensorimotor mismatch responses exist (Hertäg & Sprekeler, 2020;Mikulasch et al, 2022), it remains unclear if and how these can functionally contribute to segmentation operations. Moreover, given that a significant portion of visual input is generated by external motion, inferring the causes of optic flow becomes a two-fold problem.…”

Perceptual error based on Bayesian cue combination drives implicit motor adaptation

“…It tends to underestimate adaptation for medium-size perturbations and overestimate it for large ones (Figure 3C; see also Figure S3B for trial-by-trial fitting). Another alternative is the causal inference model, previously shown to account for nonlinearity in motor learning (Mikulasch et al, 2022;Wei & Körding, 2009) .…”

“…Another alternative is the causal inference model, previously shown to account for nonlinearity in motor learning (Mikulasch et al, 2022;Wei & Körding, 2009). Although this model has been suggested for implicit adaptation (Tsay, Avraham, et al, 2021), it fails to reproduce the observed concave adaptation pattern (Figures S3C and 3D).…”