Seeing Through Sounds: Mapping Auditory Dimensions to Data and Charts for People with Visual Impairments

Abstract

Sonification can be an effective medium for people with visual impairments to understand data in visualizations. However, there are no universal design principles that apply to various charts that encode different data types. Towards generalizable principles, we conducted an exploratory experiment to assess how different auditory channels (e.g., pitch, volume) impact the data and visualization perception among people with visual impairments. In our experiment, participants evaluated the intuitiveness and accuracy of the mapping of auditory channels on different data and chart types. We found that participants rated pitch to be the most intuitive, while the number of tappings and the length of sounds yielded the most accurate perception in decoding data. We study how audio channels can intuitively represent different charts and demonstrate that data‐level perception might not directly transfer to chart‐level perception as participants reflect on visual aspects of the charts while listening to audio. We conclude by how future experiments can be designed to establish a robust ranking for creating audio charts.

Erie: A Declarative Grammar for Data Sonification

“…For instance, Hoque et al [25] used natural sound (e.g., bird sound) to support enhanced distinction between categorical values. Wang et al [56] show that BVI readers find certain audio channels to be more intuitive given visual encodings (e.g., pitch for bar heights) and given data type (e.g., quantitative, ordinal). In their experiment, participants indicated a need for an overview of auditory scales [56].…”

“…Wang et al [56] show that BVI readers find certain audio channels to be more intuitive given visual encodings (e.g., pitch for bar heights) and given data type (e.g., quantitative, ordinal). In their experiment, participants indicated a need for an overview of auditory scales [56]. Thus, a sonification grammar should be able to express such aspects of an audio graph design definition.…”

“…An ability to express diverse designs helps creators and developers to be less constrained in making their artifacts. Due to a lack of expressive tools for data sonification, however, many prior empirical works in accessible visualization rely on more hand-crafted methods (e.g., using Garage Band by Wang et al [56]) or solutionspecific approaches (e.g., Hoque et al [25]). For example, Sonification Sandbox [53]'s authoring interface for data sonifications does not support expressing a sequence or overlay of multiple sonifications.…”

Sonificación y periodismo: la representación de datos mediante sonidos

“…Existen decenas de estudios que afirman emplear unos u otros parámetros para hacer mapeos de datos (Wang et al, 2022). A pesar de ello, aún no se ha llegado a un consenso que permita formalizar equivalencias concretas, y muchos de los autores mencionados señalan que es preciso realizar pruebas hasta encontrar la forma más acertada de realizar este mapeo para cada caso concreto.…”

SeeChart: Enabling Accessible Visualizations Through Interactive Natural Language Interface For People with Visual Impairments

“…Sio et al [74] proposed a heuristics-based approach combining sonification with textual description to generate audio data narratives from a time-series dataset and Infosonics [39] presented an unconventional method to present the audio analogue of infographics combining interactive sonifications, spoken introduction, and harmonious notes. Wang et al [81] conducted experiments to find how different auditory channels affect the perception of visualization among people with visual impairments and they reported that pitch was the most instinctive channel to understand all data types (e.g., quantitative, ordinal, and nominal). However, data sonification can lead to cognitive overload [65], and integrating such techniques in online environment is challenging [43].…”