Non-visual Perception of Lines on a Multimodal Touchscreen Tablet

Abstract

While text-to-speech software has largely made textual information accessible in the digital space, analogous access to graphics still remains an unsolved problem. Because of their portability and ubiquity, several studies have alluded to touchscreens as a potential platform for such access, yet there is still a gap in our understanding of multimodal information transfer in the context of graphics. The current research demonstrates feasibility for following lines, a fundamental graphical concept, via vibrations and sounds on commercial touchscreens. Two studies were run with 21 blind and visually impaired participants ( N = 12; N = 9). The first study examined the presentation of straight, linear lines using a multitude of line representations, such as vibration-only, auditory-only, vibration lines with auditory borders, and auditory lines with vibration borders. The results of this study demonstrated that both auditory and vibratory bordered lines were optimal for precise tracing, although both vibration- and auditory-only lines were also sufficient for following, with minimal deviations. The second study examined the presentation of curving, non-linear lines. Conditions differed on the number of auditory reference points presented at the inflection and deflection points. Participants showed minimal deviation from the lines during tracing, performing nearly equally in both 1- and 3-point conditions. From these studies, we demonstrate that line following via multimodal feedback is possible on touchscreens, and we present guidelines for the presentation of such non-visual graphical concepts.

Exploration Strategies for Tactile Graphics Displayed by Electrovibration on a Touchscreen

“…We also observed that the number of touches to the corners (edges) was significantly higher (lower) for prototypical square and hexagon than their non-prototypical counterparts. Since humans can perceive horizontal lines more easily than oriented ones (Tennison and Gorlewicz, 2019), perhaps the participants in our experiments recognized the parallel horizontal edges in prototypical square and hexagon easily and focused on the corners more to differentiate them from the others.…”

“…They concluded that participants followed non-traditional haptic exploration strategies such as navigating the screen from left to right horizontally and then from top to bottom vertically to recognize the displayed item on the touchscreen. Tennison and Gorlewicz (2019) conducted experiments with visually impaired participants to investigate how they explore virtual line profiles displayed on a touchscreen using auditory and vibratory feedback. Their results suggested that vibrating lines were preferred over auditory lines.…”

“…To increase the effectiveness of these displays, it is critical to understand how we perceive tactile graphics displayed on their touch surfaces and what exploratory procedures are followed by the users to perceive them. Although, earlier research has investigated this topic for vibrotactile feedback alone (Toennies et al, 2011;Giudice et al, 2012;Awada et al, 2013;Palani and Giudice, 2014;Tennison and Gorlewicz, 2016;Tekli et al, 2018) and also the combination with auditory feedback (Poppinga et al, 2011;Giudice et al, 2012;Tennison and Gorlewicz, 2019;Gorlewicz et al, 2020), to our knowledge, no previous work has investigated it for electrovibration in depth. In comparison to vibrotactile feedback which mainly stimulates the finger in the normal direction, electrovibration modulates the sliding friction between finger and the touch surface in the tangential direction.…”

Comparing Map Learning between Touchscreen-Based Visual and Haptic Displays: A Behavioral Evaluation with Blind and Sighted Users

“…To address this limitation, a growing body of work has begun to examine the use of touch/haptic cues as an output modality for applications supporting BVI users. Some notable examples of using touchscreen-based haptics include accessing bar graphs, letter identification, and shape discrimination [6], as well as accessing multi-line data representations [7,8], recognizing shapes and patterns [9,10], and accessing maps [11][12][13][14][15].…”