Auditory-Tactile Experience of Music

Abstract

We listen to music not only with our ears. The whole body is present in a concert hall, during a rock event, or while enjoying music reproduction at home. This chapter discusses the influence of audio-induced vibrations at the skin on musical experience. To this end, sound and body vibrations were controlled separately in several psychophysical experiments. The multimodal perception of the resulting concert quality is evaluated, and the effect of frequency, intensity, and temporal variation of the vibration signal is discussed. It is shown that vibrations play a significant role in the perception of music. Amplifying certain vibrations in a concert venue or music reproduction system can improve the music experience. Knowledge about the psychophysical similarities and differences of the auditory and tactile modality help to develop perceptually optimized algorithms to generate music-related vibrations. These vibrations can be reproduced, e.g., using electrodynamic exciters mounted to the floor or seat. It is discussed that frequency shifting and intensity compression are important approaches for vibration generation.

Audio-Tactile Rendering: A Review on Technology and Methods to Convey Musical Information through the Sense of Touch

“…The book “Musical Haptics” [ 20 ], published in 2018, compiles works on haptic technology applied to DMIs and one focused on a HMP named “Auditory-Tactile Experience of Music” by Merchel and Altinsoy [ 25 ]. This work focuses on the evaluation of music listening experience with vibrotactile feedback using a haptic seat.…”

Musical Haptic Wearables for Synchronisation of Visually-impaired Performers: a Co-design Approach

“…This endeavor falls within the remits of the emerging feld of musical haptics, which relates to the application of haptics research to the musical domain [24]. Works in this feld has included the development of haptic interfaces not only for music performers (e.g., haptically-enhanced digital musical instruments [19,34] or tactile notifcation systems [12,15,27]), but also for music listeners (e.g., haptic devices aiming to enrich the musical listening experience [21,32]).…”

Vibrotactile Captioning of Musical Effects in Audio-Visual Media as an Alternative for Deaf and Hard of Hearing People: An EEG Study

“…Some approaches to the treatment of the acoustic signal for its transformation into a tactile signal have been tested, generally considering that it may not be necessary to encode all the auditory information available in the tactile channel. As the tactile frequency perception range is 5-1000 Hz versus 20-20000 Hz for the ear, it is easier to feel the sound with a large bass component, by using low-pass filters, or by removing harmonics from the fundamental frequency [28]. User feedback has been very positive when rhythmic music is translated into tactile vibration, while more harmonic music, with less marked rhythms, have been shown to produce sensations of buzz or diffused vibration [26][28] [29].…”

“…User feedback has been very positive when rhythmic music is translated into tactile vibration, while more harmonic music, with less marked rhythms, have been shown to produce sensations of buzz or diffused vibration [26][28] [29]. One promising approach to transmit tone information is to encode it in a spatial tactile dimension, by applying different frequency bands in different areas of the skin [28] [29].…”