Mental Rotation of Digitally-Rendered Haptic Objects

Abstract

Sensory substitution is an effective means to rehabilitate many visual functions after visual impairment or blindness. Tactile information, for example, is particularly useful for functions such as reading, mental rotation, shape recognition, or exploration of space. Extant haptic technologies typically rely on real physical objects or pneumatically driven renderings and thus provide a limited library of stimuli to users. New developments in digital haptic technologies now make it possible to actively simulate an unprecedented range of tactile sensations. We provide a proof-of-concept for a new type of technology (hereafter haptic tablet) that renders haptic feedback by modulating the friction of a flat screen through ultrasonic vibrations of varying shapes to create the sensation of texture when the screen is actively explored. We reasoned that participants should be able to create mental representations of letters presented in normal and mirror-reversed haptic form without the use of any visual information and to manipulate such representations in a mental rotation task. Healthy sighted, blindfolded volunteers were trained to discriminate between two letters (either L and P, or F and G; counterbalanced across participants) on a haptic tablet. They then tactually explored all four letters in normal or mirror-reversed form at different rotations (0°, 90°, 180°, and 270°) and indicated letter form (i.e., normal or mirror-reversed) by pressing one of two mouse buttons. We observed the typical effect of rotation angle on object discrimination performance (i.e., greater deviation from 0° resulted in worse performance) for trained letters, consistent with mental rotation of these haptically-rendered objects. We likewise observed generally slower and less accurate performance with mirror-reversed compared to prototypically oriented stimuli. Our findings extend existing research in multisensory object recognition by indicating that a new technology simulating active haptic feedback can support the generation and spatial manipulation of mental representations of objects. Thus, such haptic tablets can offer a new avenue to mitigate visual impairments and train skills dependent on mental object-based representations and their spatial manipulation.

Mental Rotation of Digitally-Rendered Haptic Objects by the Visually-Impaired

“…Critically, these letters could be rotated at four angles (0 • , 90 • , 180 • , 270 • ), thus obligating participants to engage in mental rotation of the presented stimuli. (Tivadar et al, 2019) results support the fact that participants successfully managed to build mental representations of these digitallypresented haptic letters, that they were then able to rotate so as to correctly determine the form. It has also been shown that mental rotation in the visually-impaired can be supported by tactile stimuli (see Prather and Sathian, 2002, for a review).…”

“…If so, this would mean that participants are able to use digital haptic information to create mental images of 2-D objects, such as letters. Specifically, we expected to see this effect for normal trained as well as new stimuli, contrary to results in sighted (Tivadar et al, 2019), where sighted participants did not perform well with untrained (new) letters. We expect this given higher tactual expertise of visually-impaired and blind participants (Goldreich and Kanics, 2003;Legge et al, 2008;Wong et al, 2011).…”

“…It has previously been shown that sighted participants can use digital haptics to create and manipulate mental representations of letters (Tivadar et al, 2019). In this study, Tivadar and colleagues tested a group of sighted subjects on a mental rotation task with digitally-rendered haptic stimuli on the same prototype as in the current study.…”

“…The apparatus was identical to that used in Tivadar et al (2019). It entailed a 7 ′′ tablet with a 1,024 × 600 pixel touchscreen.…”

“…Stimuli were identical to those employed in Tivadar et al (2019). These were 4 upper-case letters-L, P, F, and G-drawn in Paint (see e.g., Carpenter and Eisenberg, 1978; see also Figure 1).…”