Overview of auditory representations in human-machine interfaces

Csapó, Ádám; Wersényi, György

doi:10.1145/2543581.2543586

Cited by 48 publications

(22 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our design principle is to give sounds to all relevant objects in the environment. Unlike most efforts at scene sonification (Bujacz and Strumillo, 2016; Csapo and Wersenyi, 2013), our system communicates through natural language. Each object in the scene can talk to the user with a voice that comes from the object’s location.…”

Section: Resultsmentioning

confidence: 99%

Augmented Reality Powers a Cognitive Prosthesis for the Blind

Nrb

Meister

2018

Preprint

View full text Add to dashboard Cite

A non-invasive prosthesis for blind people endows objects in the environment with voices, 9 allowing a user to explore the scene, localize objects, and navigate through a building with 10 minimal training. 12 To restore vision for the blind several prosthetic approaches have been explored that convey raw 13 images to the brain. So far these schemes all suffer from a lack of bandwidth and the extensive 14 training required to interpret unusual stimuli. Here we present an alternate approach that restores 15 vision at the cognitive level, bypassing the need to convey sensory data. A wearable computer 16 captures video and other data, extracts the important scene knowledge, and conveys that through 17 auditory augmented reality. This system supports many aspects of visual cognition: from 18 obstacle avoidance to formation and recall of spatial memories, to long-range navigation. Neither 19 training nor modification of the physical environment are required: Blind subjects can navigate 20 an unfamiliar multi-story building on their first attempt. The combination of unprecedented 21 2 computing power in wearable devices with augmented reality technology promises a new era of 22 non-invasive prostheses that are limited only by software. 23 11 Abstract

show abstract

Section: Resultsmentioning

confidence: 99%

Augmented Reality Powers a Cognitive Prosthesis for the Blind

Nrb

Meister

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…For an overview of novel representations-including representations with speech-like and/or emotional characteristics (such as spearcons, spindexes, auditory emoticons and spemoticons), as well as representations used specifically for navigation and alerting information (such as musicons and morphocons)-the reader is referred to the overview provided in [8].…”

Section: Overview Of Auditory and Haptic Feedback Methodsmentioning

confidence: 99%

A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research

Csapó

Wersényi

Nagy

et al. 2015

J Multimodal User Interfaces

Self Cite

130

View full text Add to dashboard Cite

This paper summarizes recent developments in audio and tactile feedback based assistive technologies targeting the blind community. Current technology allows applications to be efficiently distributed and run on mobile and handheld devices, even in cases where computational requirements are significant. As a result, electronic travel aids, navigational assistance modules, text-to-speech applications, as well as virtual audio displays which combine audio with haptic channels are becoming integrated into standard mobile devices. This trend, combined with the appearance of increasingly user-friendly interfaces and modes of interaction has opened a variety of new perspectives for the rehabilitation and training of users with visual impairments. The goal of this paper is to provide an overview of these developments based on recent advances in basic research and application development. Using this overview as a foundation, an agenda is outlined for future research in mobile interaction design with respect to users with special needs, as well as ultimately in relation to sensor-bridging applications in general.

show abstract

“…Sonification is the use of non-speech audio to convey information. A large variety of sonification techniques exist and are used in various applications [10,3]. The following sections contain an outline of the requirements for the sonifications, followed by a description of the two sonification techniques implemented in ZebraX .…”

Section: Guiding Modes Based On Sonificationmentioning

confidence: 99%

Sonification of guidance data during road crossing for people with visual impairments or blindness

Mascetti

Picinali

Gerino

et al. 2016

International Journal of Human-Computer Studies

View full text Add to dashboard Cite

In the last years several solutions were proposed to support people with visual impairments or blindness during road crossing. These solutions focus on computer vision techniques for recognizing pedestrian crosswalks and computing their relative position from the user. Instead, this contribution addresses a different problem; the design of an auditory interface that can effectively guide the user during road crossing. Two original auditory guiding modes based on data sonification are presented and compared with a guiding mode based on speech messages.Experimental evaluation shows that there is no guiding mode that is best suited for all test subjects. The average time to align and cross is not significantly different among the three guiding modes, and test subjects 1 arXiv:1506.07272v1 [cs.HC] 24 Jun 2015 distribute their preferences for the best guiding mode almost uniformly among the three solutions. From the experiments it also emerges that higher effort is necessary for decoding the sonified instructions if compared to the speech instructions, and that test subjects require frequent 'hints' (in the form of speech messages). Despite this, more than 2/3 of test subjects prefer one of the two guiding modes based on sonification. There are two main reasons for this: firstly, with speech messages it is harder to hear the sound of the environment, and secondly sonified messages convey information about the "quantity" of the expected movement.

show abstract

Overview of auditory representations in human-machine interfaces

Cited by 48 publications

References 65 publications

Augmented Reality Powers a Cognitive Prosthesis for the Blind

Augmented Reality Powers a Cognitive Prosthesis for the Blind

A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research

Sonification of guidance data during road crossing for people with visual impairments or blindness

Contact Info

Product

Resources

About