What you feel is what you see: inverse dynamics estimation underlies the resistive sensation of a delayed cursor

Abstract

How our central nervous system (CNS) learns and exploits relationships between force and motion is a fundamental issue in computational neuroscience. While several lines of evidence have suggested that the CNS predicts motion states and signals from motor commands for control and perception (forward dynamics), it remains controversial whether it also performs the 'inverse' computation, i.e. the estimation of force from motion (inverse dynamics). Here, we show that the resistive sensation we experience while moving a delayed cursor, perceived purely from the change in visual motion, provides evidence of the inverse computation. To clearly specify the computational process underlying the sensation, we systematically varied the visual feedback and examined its effect on the strength of the sensation. In contrast to the prevailing theory that sensory prediction errors modulate our perception, the sensation did not correlate with errors in cursor motion due to the delay. Instead, it correlated with the amount of exposure to the forward acceleration of the cursor. This indicates that the delayed cursor is interpreted as a mechanical load, and the sensation represents its visually implied reaction force. Namely, the CNS automatically computes inverse dynamics, using visually detected motions, to monitor the dynamic forces involved in our actions.

State-Based Delay Representation and Its Transfer from a Game of Pong to Reaching and Tracking

“…Recent studies have reported effects of delayed visual feedback on perception, including increased mass (Honda et al 2013) or resistance (Takamuku and Gomi, 2015), that are suggestive of a mechanical system representation. The anecdotal verbal responses of our participants that the paddle is “harder to maneuver,” “sluggish,” or “mechanical” are consistent with this view and with previous reports (Smith, 1972; Vercher and Gauthier, 1992).…”

“…None of the previous studies that linked the reported effects of delayed visual feedback to a mechanical system representation (Sarlegna et al 2010; Honda et al 2013; Takamuku and Gomi, 2015; Leib et al 2017) examined them in the context of different movement frequencies or velocities. Because a mechanical system is essentially a frequency-dependent gain together with a phase shift, evaluating the frequency dependence of the representation is critical for distinguishing between the two representations.…”

“…Humans can adapt to visuomotor delays (Miall and Jackson, 2006; Botzer and Karniel, 2013) and delayed force feedback (Witney et al 1999; Levy et al 2010; Leib et al 2015; Avraham et al 2017). However, delayed feedback biases perception of impedance (Pressman et al 2007; Nisky et al 2008, 2010; Di Luca et al 2011; Kuling et al 2015; Takamuku and Gomi, 2015; Leib et al 2016), suggesting that the sensorimotor system has limited capability to realign the signals for accurate estimations of the environment (Ionta et al 2014). …”

“…Such mechanisms can be isolated only in the absence of visual feedback. Previous studies suggested the Time-based (Rohde et al 2014; Farshchiansadegh et al 2015) and the State-based spatial shift (Smith and Bowen, 1980) and mechanical system (Sarlegna et al 2010; Takamuku and Gomi, 2015; Leib et al 2017) as candidate representation models for visuomotor delay (Rohde and Ernst, 2016). They probed delay effects on perception, or by observing action during adaptation and its aftereffects, but always with visual feedback.…”

Adaptation to a Space-variant Visuomotor Delay can Cause Neglect-like Effects on Drawing Symmetry

“…It was suggested that the misalignment between the hand and the cursor is interpreted as a mechanical load of mass (the cursor) with a spring and a damper that connects between the hand and the cursor. This model was used to explain the changes in grip forces accompanied with delayed visual feedback (Sarlegna et al, 2010), the changes in resistive sensation following adaptation to visuomotor delay (Takamuku and Gomi, 2015), and the generalization between adapting to a visuomotor delay or to a mechanical system between the hand and the cursor (Leib et al, 2017). In addition, representation of mechanical system also explains the changes in movements after adapting to force field (Wang et al, 2001).…”