Sensorimotor adaptation compensates for distortions of 3D shape information

Abstract

Visual perception often fails to recover the veridical 3D shape of objects in the environment due to ambiguity and variability in the available depth cues. However, we rely heavily on 3D shape estimates when planning movements, for example reaching to pick up an object from a slanted surface. Given the wide variety of distortions that can affect 3D perception, how do our actions remain accurate across different environments? One hypothesis is that the visuomotor system performs selective filtering of 3D information to minimize distortions. Indeed, some studies have found that actions appear to preferentially target stereo information when it is put in conflict with texture information. However, since these studies analyze averages over multiple trials, this apparent preference could be produced by sensorimotor adaptation. In Experiment 1, we create a set of cue-conflict stimuli where one available depth cue is affected by a constant bias. Sensory feedback rapidly aligns the motor output with physical reality in just a few trials, which can make it seem as if action planning selectively relies on the reinforced cue—yet no change in the relative influences of the cues is necessary to eliminate the constant errors. In contrast, when one depth cue becomes less correlated with physical reality, variable movement errors will occur, causing canonical adaptation to fail as the opposite error corrections cancel out. As a result, canonical adaptation cannot explain the preference for stereo found in studies with variable errors. However, Experiment 2 shows that persistent errors can produce a novel form of adaptation that gradually reduces the relative influence of an unreliable depth cue. These findings show that grasp control processes are continuously modified based on sensory feedback to compensate for both biases and noise in 3D visual processing, rather than having a hardwired preference for one type of depth information.

Depth cue reweighting requires altered correlations with haptic feedback

“…We regressed the MGA, our kinematic measure, against the simulated stereo and texture depths, taking the regression slope as an indicator of the influence of each cue. Our results show that cue reweighting reliably occurred when a single depth cue suddenly became uncorrelated with haptic feedback (Experiments 1 and 2), similar to our findings in another study looking at two-finger placement on slanted surfaces (experiment 2 of Cesanek et al, 2019). However, when the depth specified by a particular cue was biased to consistently under-or overestimate physical object depth, motor outputs were uniformly shifted to accurately target the physical depth, but cue reweighting was absent (Experiment 2).…”

“…Even in the very first bin of Adaptation (six trials), grasp planning had already compensated for most or all of the change in the haptic feedback. Originally, we expected to observe a more gradual shift of the MGA, as participants in previous grasp adaptation experiments required approximately 10 trials to fully adapt in response to similar perturbations (Cesanek & Domini, 2017;Cesanek et al, 2019). It is likely that the inclusion of the Pretest phase between Baseline and Adaptation disrupted the typical time course.…”

“…In these experiments, haptic feedback was manipulated to be consistent with a ''reinforced'' cue but inconsistent with one or more ''faulty'' cues, all of which are viewed simultaneously as part of a single visual surface. For example, a texture gradient specifying a 308 Citation: Cesanek, E., & Domini, F. (2019). Depth cue reweighting requires altered correlations with haptic feedback.…”

“…As motivation for our hypothesis, consider that when a depth cue is affected by a constant bias, but its reliability has not changed, other learning processes are available to resolve the resulting error signals. For instance, simply shifting the motor output via sensorimotor adaptation is a particularly efficient way to resolve movement errors resulting from a constant bias in one depth cue (Cesanek, Taylor, & Domini, 2019). Likewise, another learning process known as cue recalibration, which shifts the mapping of individual sensory signals onto world property estimates, would suffice to restore internal consistency in the face of a constant bias (Adams, Banks, & van Ee, 2001;Atkins, Jacobs, & Knill, 2003;Zaidel, Turner, & Angelaki, 2011).…”

Cue-selective adaptation operates on a separable encoding of stereo and texture information

“…shifting of the MGAs, as participants in previous grasp adaptation experiments required 26 approximately ten trials to fully adapt in response to similar perturbations (Cesanek & Domini,27 2017; Cesanek et al, 2019). It is likely that the inclusion of the Pre-test phase between Baseline 28…”