Designing the Model Human Cochlea: An Ambient Crossmodal Audio-Tactile Display

Abstract

We present a model human cochlea (MHC), a sensory substitution technique and system that translates auditory information into vibrotactile stimuli using an ambient, tactile display. The model is used in the current study to translate music into discrete vibration signals displayed along the back of the body using a chair form factor. Voice coils facilitate the direct translation of auditory information onto the multiple discrete vibrotactile channels, which increases the potential to identify sections of the music that would otherwise be masked by the combined signal. One of the central goals of this work has been to improve accessibility to the emotional information expressed in music for users who are deaf or hard of hearing. To this end, we present our prototype of the MHC, two models of sensory substitution to support the translation of existing and new music, and some of the design challenges encountered throughout the development process. Results of a series of experiments conducted to assess the effectiveness of the MHC are discussed, followed by an overview of future directions for this research.

Synchronizing to auditory and tactile metronomes: a test of the auditory-motor enhancement hypothesis

“…All stimuli were presented through voice coils embedded in a form-fitting chair (Emoti-Chair; Karam, Russo, & Fels, 2009). The voice coils were one inch in diameter and arranged in two columns of eight so that they made contact with the left and right sides of the lumbar region of the back (see Fig.…”

Auditory-Tactile Experience of Music

“…For example, it would be possible to transform the pitch of a melody into the location of vibration along the forearm, tongue, or back using multiple vibration actuators. This frequency-to-place transformation approach is usually applied in the context of tactile hearing aids, in which the tactile channel is used to replace the corrupt auditory perception [20,40]. However, in such sensory substitution systems, the transformation code needs to be learned.…”

Design of Vibrotactile Feedback and Stimulation for Music Performance

“…Karam et al [27] developed a general-purpose interface in the form of an augmented chair (the Emoti-Chair) embedded with an array of eight speakers disposed along the back. The authors' aim was to create a display for deaf people to enjoy music through vibrations.…”

“…The authors' aim was to create a display for deaf people to enjoy music through vibrations. They developed the Model Human Cochlea [26]-a sensory substitution model of the cochlear critical band filter on the back-and mapped different frequency bands of a musical track, rescaled to fit into the frequency range of sensitivity of the skin (see Sect. 4.2), to each of the speakers on the chair.…”