Multimodal 'eyes-free' interaction techniques for wearable devices

Brewster, Stephen; Lumsden, Jo; Bell, Marek; Hall, Malcolm; Tasker, Stuart

doi:10.1145/642693.642694

Cited by 57 publications

(66 citation statements)

References 4 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Innovation in these areas has explored usage of sensory modalities other than vision -for example, speech recognition 77 , non-speech auditory feedback 17 , haptic (touch-based) feedback 18 , and multimodal input 105,76 (which combines different sensory modalities) -to reduce dependence on visual interaction 19,107,21 . Recent advances in the likes of vibrotactile, text-to-speech (TTS), and gestural recognition systems have consequently opened up scope for increased accessibility to devices for persons with visual impairment.…”

Section: Mobile Devices Made Accessible For Visually-impaired Usersmentioning

confidence: 99%

See 1 more Smart Citation

Mobile assistive technologies for the visually impaired

Hakobyan

Lumsden

O’Sullivan

et al. 2013

Survey of Ophthalmology

194

108

View full text Add to dashboard Cite

This is the accepted version of the paper.This version of the publication may differ from the final published version. Permanent repository link AbstractThere are around 285 million visually-impaired people worldwide, and around 370,000 people are registered as blind or partially sighted in the UK A . On-going advances in information technology (IT) are increasing the scope for IT-based mobile assistive technologies to facilitate the independence, safety, and improved quality of life of the visually impaired. Research is being directed at making mobile phones and other handheld devices accessible via our haptic (touch) and audio sensory channels. We review research and innovation within the field of mobile assistive technology for the visually impaired and, in so doing, highlight the need for successful collaboration between clinical expertise, computer science, and domain users to realize fully the potential benefits of such technologies. We initially reflect on research which has been conducted to make mobile phones more accessible to people with vision loss. We then discuss innovative assistive applications designed for the visually impaired that are either delivered via mainstream devices and can be used while in motion (e.g., mobile phones) or are embedded within an environment that may be in motion (e.g., public transport) or within which the user may be in motion (e.g., smart homes).

show abstract

Section: Mobile Devices Made Accessible For Visually-impaired Usersmentioning

confidence: 99%

“…Brewster et al 19 proposed two novel solutions for eyes-free, mobile device use. The first presented information items to users via a 3D radial pie menu.…”

Section: Mobile Devices Made Accessible For Visually-impaired Usersmentioning

confidence: 99%

Mobile assistive technologies for the visually impaired

Hakobyan

Lumsden

O’Sullivan

et al. 2013

Survey of Ophthalmology

194

108

View full text Add to dashboard Cite

show abstract

“…229 Non-visual interfaces, particularly audio display 230 interfaces have been shown to be effective in improv-231 ing interaction and integration within existing physical 232 contexts. For example, Brewster and Pirhonen [21,22] 233 have explored the combination of gesture and audio 234 display that allows for complicated interaction with 235 mobile devices while people are in motion. The Audio 236 Aura project [23] explores how to better connect hu-237 man activity in the physical world with virtual infor-238 mation through use of audio display.…”

Section: U N C O R R E C T E D P R O O Fmentioning

confidence: 99%

“…Switching between the stereo channels created 686 localization: we used the left channel audio for the left, 687 right channel audio for the right, and both channels for 688 the center. It is an egocentric [22] spatial structure that 689 allowed the three prefaces to be distinguishable and an 690 underlying content categorization structure to exist. 691 The spatialization was mapped to the tangible interface 692 for selection.…”

mentioning

confidence: 99%

Situated play in a tangible interface and adaptive audio museum guide

Wakkary

Hatala

2006

Pers Ubiquit Comput

View full text Add to dashboard Cite

68 Abstract This paper explores the design issues of 9 situated play within a museum through the study of a 10 museum guide prototype that integrates a tangible 11 interface, audio display, and adaptive modeling. We 12 discuss our use of design ethnography in order to sit-13 uate our interaction and to investigate the liminal and 14 engagement qualities of a museum visit. The paper 15 provides an overview of our case study and analysis of 16 our user evaluation. We discuss the implications 17 including degrees of balance in the experience design 18 of play in interaction; the challenge in developing a 19 discovery-based information model, and the need for a 20 better understanding of the contextual aspects of tan-21 gible user interfaces (TUIs). We conclude that learning 22 effectiveness and functionality can be balanced pro-23 ductively with playful interaction through an adaptive 24 audio and TUI if designers balance the engagement 25 between play and the environment, and the space be-26 tween imagination and interpretation that links the 27 audio content to the artifacts.

show abstract

“…Switching between the stereo channels created localization: we used the 436 left channel audio for the left, right channel audio for the right, and both channels 437 for the center. It is a simple egocentric (Brewster et al, 2003) spatial structure that 438 allows the three prefaces to be distinguishable and an underlying content categori-439 zation structure to exist. The spatialization was mapped to the tangible user inter-440 face for selection.…”

mentioning

confidence: 99%

Ontology-Based User Modeling in an Augmented Audio Reality System for Museums

Hatala

Wakkary

2005

User Model User-Adap Inter

View full text Add to dashboard Cite

Abstract. Ubiquitous computing is a challenging area that allows us to further our under-7 standing and techniques of context-aware and adaptive systems. Among the challenges is 8 the general problem of capturing the larger context in interaction from the perspective 9 of user modeling and human-computer interaction (HCI). The imperative to address this 10 issue is great considering the emergence of ubiquitous and mobile computing environments. 11This paper provides an account of our addressing the specific problem of supporting func-12 tionality as well as the experience design issues related to museum visits through user mod-13 eling in combination with an audio augmented reality and tangible user interface system. 14 This paper details our deployment and evaluation of ec (h) Introduction 25Fundamental to human-computer interaction (HCI) is the design of interactive 26 systems that support people's goals and respond to individual backgrounds. In 27 ubiquitous computing it is equally important to consider the influence of context 28 on people's interactions and experiences. The intent is, as Fischer argues "to say 29 the 'right' thing at the 'right' time in the 'right' way" (Fischer, 2001). A critical 30 factor in ubiquitous computing is that what is perceived as "right" is largely med-31 iated by the context within which the users find themselves. 32In the area of user-adapted interaction, user modeling has attempted to address 33 many issues related to HCI. Fischer provides a clear account of the successes and 34 * This paper has not been submitted elsewhere in identical or similar form, nor will it be during the first three months after its submission to UMUAI. is the general problem of capturing the larger context in interaction (see Fischer, 36 2001, pp. 80-81). The imperative to address this issue is great considering the 37 emergence of ubiquitous and mobile computing environments. This paper provides 38 an account of addressing the specific problem of supporting functionality as well 39 as the experience design issues related to museum visits through user modeling in 40 combination with an audio augmented reality and tangible user interface system. 41We developed and tested a museum guide prototype, known as ec(h)o in order 42 to research interaction design, user modeling, and adaptive information retrieval 43 approaches that respond to the richness of a museum visit and the museum con-44 text. 45Our aim is to support the limited input common to tangible user interfaces 46 while maintaining rich and adaptive information output via a three-dimensional 47 audio display. We believe an integrated modeling technique that is weighted toward 48 modeling of implicit communication works well with a tangible user interface in 49 creating a playful and discovery-rich experience. We believe this approach com-50 bined with ontologies and a rule-based system for information retrieval provides 51 a richness of information that is responsive to the context and unique aspects of 52 the museum visitor's interaction. 5...

show abstract

Multimodal 'eyes-free' interaction techniques for wearable devices

Cited by 57 publications

References 4 publications

Mobile assistive technologies for the visually impaired

Mobile assistive technologies for the visually impaired

Situated play in a tangible interface and adaptive audio museum guide

Ontology-Based User Modeling in an Augmented Audio Reality System for Museums

Contact Info

Product

Resources

About