Sonification: Review of Auditory Display Solutions in Electronic Travel Aids for the Blind

Bujacz, Michal; Strumiłło, Paweł

doi:10.1515/aoa-2016-0040

Cited by 30 publications

(17 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our design principle is to give sounds to all relevant objects in the environment. Unlike most efforts at scene sonification (Bujacz and Strumillo, 2016; Csapo and Wersenyi, 2013), our system communicates through natural language. Each object in the scene can talk to the user with a voice that comes from the object’s location.…”

Section: Resultsmentioning

confidence: 99%

Augmented Reality Powers a Cognitive Prosthesis for the Blind

Nrb

Meister

2018

Preprint

View full text Add to dashboard Cite

A non-invasive prosthesis for blind people endows objects in the environment with voices, 9 allowing a user to explore the scene, localize objects, and navigate through a building with 10 minimal training. 12 To restore vision for the blind several prosthetic approaches have been explored that convey raw 13 images to the brain. So far these schemes all suffer from a lack of bandwidth and the extensive 14 training required to interpret unusual stimuli. Here we present an alternate approach that restores 15 vision at the cognitive level, bypassing the need to convey sensory data. A wearable computer 16 captures video and other data, extracts the important scene knowledge, and conveys that through 17 auditory augmented reality. This system supports many aspects of visual cognition: from 18 obstacle avoidance to formation and recall of spatial memories, to long-range navigation. Neither 19 training nor modification of the physical environment are required: Blind subjects can navigate 20 an unfamiliar multi-story building on their first attempt. The combination of unprecedented 21 2 computing power in wearable devices with augmented reality technology promises a new era of 22 non-invasive prostheses that are limited only by software. 23 11 Abstract

show abstract

Section: Resultsmentioning

confidence: 99%

Augmented Reality Powers a Cognitive Prosthesis for the Blind

Nrb

Meister

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Bujacz also used the relationship between stereo vision, ultrasonic sensing, and the activation of modulated sounds at different frequencies to indicate the position of obstacles placed along the way [17]. The authors were inspired by sonar-based models to provide feedback to the user, using a five-tone full-scale sound bank and two octaves of these tones played by a piano to represent certain positions of the test scenario.…”

Section: State Of the Artmentioning

confidence: 99%

“…HEYA et al, 2018 [10] SLAM KNN Sound KITT et al, 2010 [11] Proximity Method Kalman filter Visual XUE et al, 2016 [12] Proximity Method RANSAC Visual PRESTI et al, 2019 [15] Proximity Method Weighted average Polytonic MASSICETI et al, 2018 [16] Proximity Method KNN humming sound BUJACZ et al, 2016 [17] Proximity Method Particle filter humming sound ALCANTARILLA et al, 2012 [14] SLAM Weighted average Visual CHEN et al, 2014 [6] PDR Kalman filter Visual…”

Section: Author Navigation Algorithms Data Fusion Alert Typementioning

confidence: 99%

Audio Guide for Visually Impaired People Based on Combination of Stereo Vision and Musical Tones

Simões

Silva²,

Pio³

et al. 2019

Sensors

View full text Add to dashboard Cite

Indoor navigation systems offer many application possibilities for people who need information about the scenery and the possible fixed and mobile obstacles placed along the paths. In these systems, the main factors considered for their construction and evaluation are the level of accuracy and the delivery time of the information. However, it is necessary to notice obstacles placed above the user’s waistline to avoid accidents and collisions. In this paper, different methodologies are associated to define a hybrid navigation model called iterative pedestrian dead reckoning (i-PDR). i-PDR combines the PDR algorithm with a Kalman linear filter to correct the location, reducing the system’s margin of error iteratively. Obstacle perception was addressed through the use of stereo vision combined with a musical sounding scheme and spoken instructions that covered an angle of 120 degrees in front of the user. The results obtained in the margin of error and the maximum processing time are 0.70 m and 0.09 s, respectively, with obstacles at ground level and suspended with an accuracy equivalent to 90%.

show abstract

“…Surveys conducted with the visually impaired (Marston, ) have indicated the three most problematic challenges faced by blind people: (1) independent safe mobility, (2) spatial orientation and navigation, (3) access to visual information (text and graphics). The authors’ previous experience has been with devices that address the first two problems, mainly in the form of electronic travel aids (Bujacz & Strumillo, ) or navigation software (Baranski & Strumillo, ; Skulimowski, Korbel, & Wawrzyniak, ). In this project, however, we attempt to address the third major problem and focus on interactive auditory presentation of images for the use in education of blind children.…”

Section: Introductionmentioning

confidence: 99%

Interactive sonification of images in serious games as an education aid for visually impaired children

Radecki¹,

Bujacz²,

Skulimowski³

et al. 2019

Brit J Educational Tech

Self Cite

View full text Add to dashboard Cite

The paper presents an application for interactive sonification of images intended for use on mobile devices in education of blind children at elementary school level. The paper proposes novel sonification algorithms for converting colour and grayscale images into sound. The blind user can interactively explore image content through multi‐touch gestures and select image sub‐regions for sonification with real‐time control of the synthesized sound parameters such as amplitude, frequency and timbre. Additionally, images may contain text fields read by text‐to‐speech synthesizer. The usability of one of the proposed sonification schemes is tested by collecting data on the tracking accuracy and recognition speed of basic shapes, such as lines, curves, as well as figures and simple functions. In order to facilitate the learning process for blind children, a number of games were proposed that use the devised sonification schemes. The first game—“Hear the Invisible”—is intended for two players—one child draws a shape and the task of the other one is to guess the displayed shape by means of the available sonification methods. The second proposed game is “Follow the Rabbit” and is intended for a single player who tracks a colourful “Rabbit” that runs along a path representing a given geometric shape. The obtained results show that the proposed sonification methods allow to reach previously unattainable levels of flexibility in exploration of shapes and colours. The main application of the described interactive tool is to teach geometry and mathematics in schools for blind children. The developed games are meant to enhance the learning process and motivate the children. What is known about the topic Sonification is a generic term for data‐based non‐speech sound generation. Interactive sonification is a relatively fresh area of study, where the listener heavily influences the process of sound synthesis. Earlier studies on sonification and educational computer games have shown that it is possible to demonstrate shapes such as plots and geometry using only non‐verbal sounds. What the paper adds The paper proposes an interactive sonification algorithm that can be used on greyscale or colour images using additive synthesis and the HSV colour representation. The presented software also allows to record and analyse user interaction with the images, making it a good research tool for current and future sonification algorithms. The paper presents a pilot study with blind children using two games that improve the learning process of the proposed algorithms. Implications for practice and/or policy Once the software is openly released it can be easily installed on any Android OS device to allow blind users to analyse images by touch. The proposed games train the users the basics of the sonification algorithm in a competitive context. The software can be especially useful in education of blind children, eg, in teaching geometry, math and in training visual and spatial imagination.

show abstract

Sonification: Review of Auditory Display Solutions in Electronic Travel Aids for the Blind

Cited by 30 publications

References 22 publications

Augmented Reality Powers a Cognitive Prosthesis for the Blind

Augmented Reality Powers a Cognitive Prosthesis for the Blind

Audio Guide for Visually Impaired People Based on Combination of Stereo Vision and Musical Tones

Interactive sonification of images in serious games as an education aid for visually impaired children

Contact Info

Product

Resources

About