Audio-Tactile Rendering: A Review on Technology and Methods to Convey Musical Information through the Sense of Touch

Abstract

Tactile rendering has been implemented in digital musical instruments (DMIs) to offer the musician haptic feedback that enhances his/her music playing experience. Recently, this implementation has expanded to the development of sensory substitution systems known as haptic music players (HMPs) to give the opportunity of experiencing music through touch to the hearing impaired. These devices may also be conceived as vibrotactile music players to enrich music listening activities. In this review, technology and methods to render musical information by means of vibrotactile stimuli are systematically studied. The methodology used to find out relevant literature is first outlined, and a preliminary classification of musical haptics is proposed. A comparison between different technologies and methods for vibrotactile rendering is performed to later organize the information according to the type of HMP. Limitations and advantages are highlighted to find out opportunities for future research. Likewise, methods for music audio-tactile rendering (ATR) are analyzed and, finally, strategies to compose for the sense of touch are summarized. This review is intended for researchers in the fields of haptics, assistive technologies, music, psychology, and human–computer interaction as well as artists that may make use of it as a reference to develop upcoming research on HMPs and ATR.

Recent Developments in Haptic Devices Designed for Hearing-Impaired People: A Literature Review

“…The paper by Remache et al. [ 27 ] compiled different methods and technologies used to communicate music through touch. Meanwhile, another paper Fletcher [ 28 ] discussed studies showing that haptic stimulation can improve musical perception in people with hearing impairments.…”

Customizing and Evaluating Accessible Multisensory Music Experiences with Pre-Verbal Children—A Case Study on the Perception of Musical Haptics Using Participatory Design with Proxies

“…The potential of vibroacoustic therapy for persons with disabilities has also been stressed [26]. A review of haptic feedback in sensory substitution systems allowing persons with hearing impairments to experience music through the sense of touch, so-called Haptic Music Players (HMPs), was recently published in [27]. An HMP is a device that can process an audio signal to extract musical information and translate this information into vibratory signals.…”

Effect of audio-tactile congruence on vibrotactile music enhancement

“…The Emoti-Chair can therefore have frequency congruence and acoustic frequency to tactile location congruence in the frequency model. (For a full review on devices which convey music through haptics, see Remache-Vinueza et al, 2021).…”