Designing Motion Gesture Interfaces in Mobile Phones for Blind People

Abstract

Despite the existence of advanced functions in smartphones, most blind people are still using old-fashioned phones with familiar layouts and dependence on tactile buttons. Smartphones support accessibility features including vibration, speech and sound feedback, and screen readers. However, these features are only intended to provide feedback to user commands or input. It is still a challenge for blind people to discover functions on the screen and to input the commands. Although voice commands are supported in smartphones, these commands are difficult for a system to recognize in noisy environments. At the same time, smartphones are integrated with sophisticated motion sensors, and motion gestures with device tilt have been gaining attention for eyes-free input. We believe that these motion gesture interactions offer more efficient access to smartphone functions for blind people. However, most blind people are not smartphone users and they are aware of neither the affordances available in smartphones nor the potential for interaction through motion gestures. To investigate the most usable gestures for blind people, we conducted a user-defined study with 13 blind participants. Using the gesture set and design heuristics from the user study, we implemented motion gesture based interfaces with speech and vibration feedback for browsing phone books and making a call. We then conducted a second study to investigate the usability of the motion gesture interface and user experiences using the system. The findings indicated that motion gesture interfaces are more efficient than traditional button interfaces. Through the study results, we provided implications for designing smartphone interfaces.

Usability Studies on Mobile User Interface Design Patterns: A Systematic Literature Review

“…The first one was to study factors that affected input accuracy, for instance, visual distraction, user's sensorimotor coordination, blindness, and aging [87][88][89][90][91]. The second one was to improve input accuracy, for instance, changing keyboard layouts and increasing input feedback [92,93].…”

“…A common practice is to use audio augmentation techniques, such as voice-over (a.k.a. screen reader) and speech recognition [90,91]. However, audio augmentation techniques can be interfered with by background noise [90], and the performance of audio augmentation techniques is also influenced by algorithm recognition accuracy.…”

“…screen reader) and speech recognition [90,91]. However, audio augmentation techniques can be interfered with by background noise [90], and the performance of audio augmentation techniques is also influenced by algorithm recognition accuracy. Recent studies have designed and developed a gesture-based set of commands [90] and tangible bendable gestures for blind users [91].…”

“…However, audio augmentation techniques can be interfered with by background noise [90], and the performance of audio augmentation techniques is also influenced by algorithm recognition accuracy. Recent studies have designed and developed a gesture-based set of commands [90] and tangible bendable gestures for blind users [91]. The studies highlighted that the gestures designed for blind users should be logical and easy to learn and remember, as blind users rely much on their memories [91].…”

Measuring the effect of public vs private context on the kinematics of smartphone motion gestures

“…Des travaux récents ont exploré les gestes comme modalité d'interaction pour les smartphones [3,13]. En supposant que ces gestes peuvent être appris et reconnus de manière fiable, ces travaux montrent que les gestes peuvent être utiles pour invoquer des commandes lorsque l'écran du smartphone n'est pas disponible.…”