Sensori-"motor" coupling by observed and imagined movement

Abstract

Sensory systems are associated with motor systems for perception. In the absence of motor control over the orientation of the sensory input, a person may have no idea from where the information is coming, and thus no ability to locate it in space. Sensory substitution studies have demonstrated that the sensory part of a sensory-motor loop can be provided by artificial receptors leading to a brain-machine interface (BMI). We now propose that the motor component of the sensory-motor coupling can be replaced by a "virtual" movement. We suggest that it is possible to progress to the point where predictable movement, not observed except for some sign of its initiation, could be imagined and by that means the mental image of movement could substitute for the motor component of the loop. We further suggest that, due to the much faster information transmission of the skin than the eye, innovative information presentation, such as fast sequencing and time division multiplexing can be used to partially compensate for the relatively small number of tactile stimulus points in the BMI. With such a system, incorporating humans-inthe-loop for industrial applications could result in increased efficiency and humanization of tasks that presently are highly stressful.

Tactile Sensory Substitution Studies

“…We recently proposed that the motor component of the sensorimotor coupling can be replaced by a “virtual” movement. We have suggested that it is possible to progress to the point where predictable movement, not observed except for some sign of its initiation, could be imagined and by that means the mental image of movement could substitute for the motor component of the loop (Bach‐y‐Rita and Kercel, 2003). We further suggested that, due to the much faster information transmission of the skin than the eye, innovative information presentation, such as fast sequencing and time division multiplexing can be used to partially compensate for the relatively small number of tactile stimulus points in the BMI.…”

Some Radical Implications of Bach-Y-Rita's Discoveries

“…The nearly instantaneous recovery of balance by his bilateral vestibular dysfunction patients is little short of amazing. As remarkable as these results have proven to be, as Bach-y-Rita himself points out, the restoration of lost senses is only the beginning of the possibilities that can flow from technologies that harness the untapped potential of the brain [7,8].…”

“…In this case as in many instances of incipient catastrophe, "intuitive" responses make a difference between life and death [8]. For more than two hours [42], confusing, incorrect and contradictory (pressurizer level reading pegged-out on the high end and pressure reading absurdly low) instrument readings had baffled all the operators present [18]; they were religiously complying with procedures that never anticipated the combination of failures that actually occurred.…”

Brain-Computer Interface Research

“…In a seminal sensory substitution experiment, blind subjects were taught to detect visual objects using a system that translated video input from a camera into a matrix of tactile inputs on the subject's back (Bach-y-Rita et al 1969;Bach-y-Rita and Kercel 2003). Users of this system had great success, even learning to recognize faces and partially occluded objects, and perceived "the external localization of stimuli," meaning that the objects they sensed seemed to come directly from the camera rather than having to be interpreted from a sense of tingling or touch on their backs (Bach-y-Rita et al 1969;Bach-y-Rita and Kercel 2002). Importantly, subjects only learned to distinguish objects when they were able to actively manipulate the movement and perspective of the camera, observing the changes in feedback as a result of their actions.…”