Mental Rotation of Digitally-Rendered Haptic Objects by the Visually-Impaired

Abstract

In the event of visual impairment or blindness, information from other intact senses can be used as substitutes to retrain (and in extremis replace) visual functions. Abilities including reading, mental representation of objects and spatial navigation can be performed using tactile information. Current technologies can convey a restricted library of stimuli, either because they depend on real objects or renderings with low resolution layouts. Digital haptic technologies can overcome such limitations. The applicability of this technology was previously demonstrated in sighted participants. Here, we reasoned that visually-impaired and blind participants can create mental representations of letters presented haptically in normal and mirror-reversed form without the use of any visual information, and mentally manipulate such representations. Visually-impaired and blind volunteers were blindfolded and trained on the haptic tablet with two letters (either L and P or F and G). During testing, they haptically explored on any trial one of the four letters presented at 0 • , 90 • , 180 • , or 270 • rotation from upright and indicated if the letter was either in a normal or mirror-reversed form. Rotation angle impacted performance; greater deviation from 0 • resulted in greater impairment for trained and untrained normal letters, consistent with mental rotation of these haptically-rendered objects. Performance was also generally less accurate with mirror-reversed stimuli, which was not affected by rotation angle. Our findings demonstrate, for the first time, the suitability of a digital haptic technology in the blind and visually-impaired. Classic devices remain limited in their accessibility and in the flexibility of their applications. We show that mental representations can be generated and manipulated using digital haptic technology. This technology may thus offer an innovative solution to the mitigation of impairments in the visually-impaired, and to the training of skills dependent on mental representations and their spatial manipulation.

Mental Rotation Skill Shapes Haptic Exploration Strategies

“…coauthors [29] observed that high visuospatial imagers better recognized raised-line drawings than low visuospatial imagers. Similarly, Kalisch et al [24] found better haptic recognition of solid objects in aged persons with high cognitive abilities compared to their less-skilled peers. The fact that a visual mental rotation test positively correlated with the performance in a haptic task is also consistent with a wide literature highlighting the role of a common multimodal representation system processing both visual and haptic information [39]- [47].…”

“…Previous studies focused on how haptic perception can be exploited to generate, manipulate and improve mental representations of spatial information. For instance, Tivadar and coauthors [23], [24] used a haptic tablet to present letters at different orientations to blindfolded sighted or blind persons that were asked to indicate if the letter was either in a normal or mirror-reversed form. Results showed higher performance for normal letters compared to mirrored letters and for trained as compared to untrained letters, thereby demonstrating mental rotation of haptic letter stimuli.…”

Digital Haptics Improve Speed of Visual Search Performance in a Dual-Task Setting

“…More extensive training would likely improve performance on the tablet task under haptic-only conditions. In fact, previous results show that after a training session of about 45 minutes on the haptic tablet, blindfolded sighted participants as well as visually impaired individuals were able to discriminate letters and to build mental representations of these letters that they then mentally manipulated in a mental rotation task 32,33 .…”

“…This type of haptic tablet uses ultrasonic vibrations to modulate the friction of a at screen under the actively exploring nger, and thus simulates the localized perception of texture 30,31 . Such devices are proving to be especially effective as sensory substitution devices for the visually-impaired, given their ability to simulate the perception of objects and their very low training requirements 32,33 . What is distinct about these technical innovations is their ability to render recognisable and manipulable textures and objects, rather than simply provide localized haptic feedback.…”

Congenitally blind adults can learn to identify face-shapes via auditory sensory substitution and successfully generalize some of the learned features

“…Mental rotation of objects has been linked to motor experience, hormones, and experience-dependent processes 36 – 38 . Previous studies showed that blind individuals, even those who acquired blindness at birth and thus have no visual experience, are able to mentally rotate objects perceived through touch and are not deficient in their mental rotation abilities compared to sighted adults performing the same task also through touch 39 – 41 . Overall, the current results strengthen previous findings concluding that both the construction of a mental image and its rotation are not limited to information acquired via the visual modality.…”