Dissociable effects of urgency and evidence accumulation during reaching revealed by dynamic multisensory integration

Abstract

When making perceptual decisions, humans combine information across sensory modalities dependent on their respective uncertainties. However, it remains unknown how the brain handles multisensory integration during movement, and which factors besides sensory uncertainty might influence the contribution of different modalities. We performed two reaching experiments on healthy adults to investigate whether movement corrections to combined visual and mechanical perturbations scale with visual uncertainty. To describe the dynamics of multimodal feedback responses, we further varied movement speed and duration of visual feedback during the movement. The results of our first experiment (N=16, 11 females) show that the contribution of visual feedback decreased with uncertainty. Interestingly, we observed a transient phase during which visual feedback responses were stronger during faster movements. In a follow-up experiment (N=16, 10 females), we found that the contribution of vision increased more quickly during slow movements when we presented the visual feedback for a longer time. Using an optimal feedback control model, we show that the increased response to visual feedback during fast movements can be explained by an urgency-dependent increase in control gains. Further, the fact that viewing duration increased the visual contributions suggests that the brain indeed performs a continuous state-estimation as expected in the optimal control model featuring a Kalman filter. Hence, both uncertainty and urgency determine how the sensorimotor system responds to multimodal perturbation during reaching control. We highlight similarities between reaching control and decision-making, both of which appear to be influenced by the accumulation of sensory evidence as well as response urgency.Significance statementThe exact time course of multisensory integration during movement, along with the factors that influence this process, still requires further investigation. Here, we tested how visual uncertainty, movement speed, and visual feedback duration influence corrective movements during reaching with combined visual and mechanical perturbations. Using an optimal feedback control model, we illustrate that the time course of multimodal feedback responses follows the predictions of a Kalman filter which continuously weighs sensory feedback and internal predictions according to their reliability. Importantly, we further show that changes in movement speed led to urgency-dependent modulations of control gains. Our results highlight connections between motor control and decision-making processes, which both depend on the accumulation of sensory evidence and response urgency.