Evaluation of psychoacoustic sound parameters for sonification

Abstract

Sonification designers have little theory or experimental evidence to guide the design of data-to-sound mappings. Many mappings use acoustic representations of data values which do not correspond with the listener's perception of how that data value should sound during sonification. This research evaluates data-to-sound mappings that are based on psychoacoustic sensations, in an attempt to move towards using data-to-sound mappings that are aligned with the listener's perception of the data value's auditory connotations. Multiple psychoacoustic parameters were evaluated over two experiments, which were designed in the context of a domain-specific problem -detecting the level of focus of an astronomical image through auditory display. Recommendations for designing sonification systems with psychoacoustic sound parameters are presented based on our results.

Investigating Perceptual Congruence between Data and Display Dimensions in Sonification

“…These parameters are responsible for a substantial part of the affect a sound has on a listener -combined, they can model the relative degree of noise annoyance [10], and roughness independently has been found to be a primary component in both natural and artificial alarm sounds [2]. Previous work found that data:sound mappings in which the acoustic parameters were based on psychoacoustic parameters were effective in the context of an astronomical image quality assessment task [14]. However, this work was limited in scope, focusing on a single use-case -there is no research that provides a number of different contexts in which psychoacoustic parameters may be more effective than acoustic parameters traditionally used in sonification such as pitch and tempo.…”

“…In a previous study, we successfully used psychoacoustic parameters to display the degree of focus of an astronomical image [14]. In this study, the acoustic parameters used in a sonification were the psychoacoustic sensation of roughnessthe subjective perception of amplitude or frequency modulation of a sound; a signal that was broadband noise at 0 % and a pure clean tone at 100 % -this utilised a potential association of noise with uncertainty and a clear tone with clarity; and a redundant combination of roughness and noise.…”

“…As shown in [14,38,43,45], the mental models or "expectancies" a listener has of how a data value should sound during sonification is an important factor in the effectiveness of the auditory display. Psychoacoustic parameters such as roughness are a strong indicator of a listener's perceived quality of a sound [49].…”

“…Likewise, the effectiveness of the noise:focus mapping in [14] suggests that listeners expect a blurry image to sound noisy when it is sonified, whereas a crisp, focused image should sound clear. Based on these two examples, it is possible that these auditory parameters could convey semantically similar variables effectively, for example stress or error instead of danger, and quality or correctness instead of blur.…”

“…Error, danger and stress were chosen as data variables, due to previous studies' suggestions that the perception of danger and error can be dependent on acoustic qualities [2,14] and that they are a general set of potential use cases that are semantically similar -all of these are generally considered negative variables, as are the acoustic parameters of roughness and noise. The experiment used a within-subjects design.…”