Interactive sonification of complex data

Pauletto, Sandra; Hunt, Andy

doi:10.1016/j.ijhcs.2009.05.006

Cited by 34 publications

(22 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this general sense, sonification has also been applied in human movement science: Age-dependent characteristics in electromyographic data have been perceived after assigning a frequency to each surface electrode placed on the quadriceps femoris and after mapping monitored activity to the amplitude (Pauletto & Hunt, 2006).…”

Section: Sonification Of Movement Variablesmentioning

confidence: 99%

Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review

et al. 2012

View full text Add to dashboard Cite

It is generally accepted that augmented feedback, provided by a human expert or a technical display, effectively enhances motor learning. However, discussion of the way to most effectively provide augmented feedback has been controversial. Related studies have focused primarily on simple or artificial tasks enhanced by visual feedback. Recently, technical advances have made it possible also to investigate more complex, realistic motor tasks and to implement not only visual, but also auditory, haptic, or multimodal augmented feedback. The aim of this review is to address the potential of augmented unimodal and multimodal feedback in the framework of motor learning theories. The review addresses the reasons for the different impacts of feedback strategies within or between the visual, auditory, and haptic modalities and the challenges that need to be overcome to provide appropriate feedback in these modalities, either in isolation or in combination. Accordingly, the design criteria for successful visual, auditory, haptic, and multimodal feedback are elaborated.

show abstract

Section: Sonification Of Movement Variablesmentioning

confidence: 99%

Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review

et al. 2012

View full text Add to dashboard Cite

show abstract

“…Several issues were considered when designing the sound‐based data representations. These included mapping (which sound attributes are used), polarity (direction of change in the sound attribute), and scaling (how much change in the sound is needed to convey a given change in the referent); the structure of the data generating the sound [noisy or random (in the current study)]; constant, linearly ascending, discontinuous, or interrupted (Pauletto & Hunt 2009) and detectability and discriminability, interactivity, individual differences, training, concurrent sounds, and delivery of audio through hardware and software (Nees & Walker 2009).…”

Section: Literature Reviewmentioning

confidence: 99%

Enabling people who are blind to experience science inquiry learning through sound‐based mediation

Levy

Lahav

2011

Computer Assisted Learning

View full text Add to dashboard Cite

This paper addresses a central need among people who are blind, access to inquiry‐based science learning materials, which are addressed by few other learning environments that use assistive technologies. In this study, we investigated ways in which learning environments based on sound mediation can support science learning by blind people. We used NetLogo, a multi‐agent programmable modeling environment that is widely used for learning about complex systems. In order to provide blind people with access to such models, we used a component that supports sound‐based mediation. The sound‐based mediation provided real‐time information regarding objects' speed, location, and interactions with other objects. We examined blind people's learning about a chemical system of contained gas particles. The study employs a pre‐test intervention–post‐test design. Four adults participated individually in the study. They achieved most referent‐representation connections; their scientific conceptual knowledge became more specific and aligned with scientific knowledge; and their systems reasoning showed greater discrimination and relation between components. Discussion addresses learning with sound‐based mediation in broader terms and suggests further research into the potential of this unique type of low‐cost learning environment to assist blind people in their science learning.

show abstract

“…providing a particular input based on a certain stimulus) have shown that a stimulus represented in multiple modalities simultaneously can be detected at lower thresholds, faster and more accurately than when information is provided separately in each modality [14,18]. This characteristic has been recently explored in the domain of complex data representation and interpretation where studies found that supplemental sonification, e.g., in addition to visual representation is beneficial [22,25].…”

Section: Introductionmentioning

confidence: 99%

Supplemental sonification of a bingo game

Ramos¹,

Folmer

2011

Proceedings of the 6th International Conference on Foundations of Digital Games

View full text Add to dashboard Cite

Visual cues are typically used in video games to indicate to the player what input to provide and when. Cues represented in multiple modalities that are presented simultaneously can be detected at lower thresholds, faster and more accurately than when presented separately in each modality. This characteristic has not been explored in playing video games to reduce errors. This paper explores the use of supplemental audio feedback to reduce errors in playing Bingo, a game which is typically played in crowded and noisy environments by a demographic, which -due to their age-are more likely to suffer from sensory impairments such as low vision or hearing impairments. A user study explored three different types of sonification (pitch, timbre, and audio icons) versus using no sonification and found that supplemental sonification using timbre or audio icons significantly reduces player's errors.

show abstract

Interactive sonification of complex data

Cited by 34 publications

References 10 publications

Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review

Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review

Enabling people who are blind to experience science inquiry learning through sound‐based mediation

Supplemental sonification of a bingo game

Contact Info

Product

Resources

About