Learning New Sensorimotor Contingencies: Effects of Long-Term Use of Sensory Augmentation on the Brain and Conscious Perception

Abstract

Theories of embodied cognition propose that perception is shaped by sensory stimuli and by the actions of the organism. Following sensorimotor contingency theory, the mastery of lawful relations between own behavior and resulting changes in sensory signals, called sensorimotor contingencies, is constitutive of conscious perception. Sensorimotor contingency theory predicts that, after training, knowledge relating to new sensorimotor contingencies develops, leading to changes in the activation of sensorimotor systems, and concomitant changes in perception. In the present study, we spell out this hypothesis in detail and investigate whether it is possible to learn new sensorimotor contingencies by sensory augmentation. Specifically, we designed an fMRI compatible sensory augmentation device, the feelSpace belt, which gives orientation information about the direction of magnetic north via vibrotactile stimulation on the waist of participants. In a longitudinal study, participants trained with this belt for seven weeks in natural environment. Our EEG results indicate that training with the belt leads to changes in sleep architecture early in the training phase, compatible with the consolidation of procedural learning as well as increased sensorimotor processing and motor programming. The fMRI results suggest that training entails activity in sensory as well as higher motor centers and brain areas known to be involved in navigation. These neural changes are accompanied with changes in how space and the belt signal are perceived, as well as with increased trust in navigational ability. Thus, our data on physiological processes and subjective experiences are compatible with the hypothesis that new sensorimotor contingencies can be acquired using sensory augmentation.

Neuromorphic Vibrotactile Stimulation of Fingertips for Encoding Object Stiffness in Telepresence Sensory Substitution and Augmentation Applications

“…Vibrotactile stimulation can also be widely used in the field of non-invasive feedback in amputees and sensory augmentation for healthy subjects in applications such as virtual reality, gaming, rehabilitation, navigation, rescue and remote control of robots [ 27 , 28 , 29 , 30 , 31 ]. Examples of tactile devices for sensory augmentation providing whisker-type distance information were developed in the shape of head mounted systems [ 54 , 55 , 56 ] or belts [ 57 , 58 ] to improve navigation, as well as handheld systems for the detection of distances [ 58 ].…”

Can Limitations of Visuospatial Attention Be Circumvented? A Review

“…The possibility to circumvent limitations of visuospatial attention is also relevant for many real-world tasks that require visuospatial attention such as car-driving (Spence and Read, 2003 ; Kunar et al, 2008 ; Spence and Ho, 2012 ), air-traffic control (Giraudet et al, 2014 ), aviation (Nikolic et al, 1998 ; Sklar and Sarter, 1999 ), navigation (Nagel et al, 2005 ; Kaspar et al, 2014 ; König et al, 2016 ), or rehabilitation (Johansson, 2012 ; Maidenbaum et al, 2014 ). Notably, for applying findings to real-world tasks additional factors such as how much the task was practiced (Ruthruff et al, 2001 ; Chirimuuta et al, 2007 ) or memorized (Matusz et al, 2017 ) should be taken into account as real-world tasks are often highly practiced and memorized.…”

Embodied Spatial Knowledge Acquisition in Immersive Virtual Reality: Comparison to Map Exploration

“…Direct experience supports route knowledge in relation to an egocentric reference frame whereas spatial learning with a map heightens survey knowledge in relation to an allocentric reference frame (Frankenstein, Mohler, Bülthoff, & Meilinger, 2012;Meilinger et al, 2013;Meilinger et al, 2015;Montello et al, 2004;Richardson et al, 1999;Shelton & McNamara, 2004;Taylor et al, 1999;Thorndyke & Hayes-Roth, 1982). Due to previous research using the feelSpace augmentation device that gives information about magnetic north (Kaspar et al, 2014;König et al, 2016), we designed the tasks investigated in this study with respect to an allocentric reference frame. The tasks that tested orientation of houses' facing directions towards north or relative orientation between two houses' facing directions directly used knowledge of cardinal directions.…”

“…Another group of participants explored the city in VR with the feelSpace belt, a device supplying information about magnetic north (Kaspar, König, Schwandt, & König, 2014;König et al, 2016;Nagel, Carl, Kringe, Märtin, & König, 2005). However, the data of this group are not covered in the present report.…”