A Wearable Multimodal Interface for Exploring Urban Points of Interest

Abstract

Locating points of interest (POIs) in cities is typically facilitated by visual aids such as paper maps, brochures, and mobile applications. However, these techniques require visual attention, which ideally should be on the surroundings. Non-visual techniques for navigating towards specific POIs typically lack support for free exploration of the city or more detailed guidance. To overcome these issues, we propose a multimodal, wearable system for alerting the user of nearby recommended POIs. The system, built around a tactile glove, provides audio-tactile cues when a new POI is in the vicinity, and more detailed information and guidance if the user expresses interest in this POI. We evaluated the system in a field study, comparing it to a visual baseline application. The encouraging results show that the glovebased system helps keep the attention on the surroundings and that its performance is on the same level as that of the baseline.

Designing for Mixed Reality Urban Exploration

“…Then, it engaged in several research through design activities [13], to produce research prototypes that were tested with potential users in a historical city. The prototypes were used as probes to investigate how we could concretize the envisioned design principles into technological features that would successfully address the typical problems faced in MRUE [14][15][16][17][18][19][20][21]. The investigation led to the integration of different modalities, such as VR, AR, and haptics-audio interfaces, as well as advanced features such as personalized recommendations, social exploration, and itinerary management into the same mobile application running on a tablet.…”

“…Vibrotactile guidance cues on the hand then lead users to point in the direction of the recommended POI. We conducted a comparative study with a context-aware mobile app and found that although experiencing similar performance of different evaluation metrics, smartphone users spent on average 70% of their time looking at the screen when exploring the urban area, whereas users wearing our haptic glove were able to have a good exploration experience while leaving their visual attention on the surroundings [16].…”

“I was Holding a Magic Box”: Investigating the Effects of Private and Projected Displays in Outdoor Heritage Walks

“…In this work, we focus primarily on the effects that alternative display technologies can have on the experience of participants, with respect to their engagement with the physical and social environment. Prior work includes several attempts at revealing content directly in the physical space of the heritage sites through different types of digital augmentation and interaction techniques to overcome the head-down interaction [31,58,59]. Motivated by such attempts we further consider the effects of display technologies that allow shared views of content, as well as content blended with the physical environment, and their potential in enabling holistic awareness of surrounding, corporeal engagement with the heritage site, and social interaction.…”

A Review of Wrist-Worn Wearable: Sensors, Models, and Challenges

“…(i) Smartwatch (ii) Smart eyewear (e.g., smart glasses and headmounted displays) [14] (iii) Egocentric vision devices [15] (iv) Light-based devices (e.g., LED) [16][17][18][19][20][21][22][23][24] (v) Fabrics, textiles, and skin-based devices [25][26][27][28] (vi) Tactile gloves [29] (vii) Hair and nail-based devices [30] (viii) Magnetic inputs (e.g., Google cardboard) [31,32] These wearable devices' main challenges are networking, power and heat, display, and mobile input. All of them should be affordable for low-income earners and small in size and consume a small amount of battery power [33].…”