Designing a model human cochlea: issues and challenges in crossmodal audio-haptic displays

Karam, Maria; Fels, Deborah I.

doi:10.4108/icst.ambisys2008.2837

Cited by 11 publications

(15 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Results from this study are intended to contribute to future work that will investigate the use of a vibrotactile display by deaf and hard of hearing to support music listening [16], [4], [17].…”

Section: The Present Studymentioning

confidence: 94%

Vibrotactile Display of Music on the Human Back

Branje

Maksimouski²,

Karam

et al. 2010

2010 Third International Conference on Advances in Computer-Human Interactions

Self Cite

View full text Add to dashboard Cite

We present an experiment designed to reveal characteristics of a tactile display that presents vibrations representing music to the back of the body. Based on the model human cochlea, a sensory substitution system aimed at translating music into vibrations, we are investigating the use of larger contactor sizes (over 10mm in diameter) as an effective device for the detection of signals originating from music. Using the method of limits, we measured ability to discriminate the frequency of vibrotactile stimuli across a wide range of frequencies common to western classical harmonic music. Vibrotactile stimuli were presented to artificially deafened participants using a large contactor applied to the back. Between 65 Hz (C2) and 1047 Hz (C6), frequency difference limens (FDL) were consistently less than 1/3 of an octave and as small as 200 cents. These findings suggest that vibrotactile information can be used to support the experience of music even in the absence of sound, and that voice coils are effective in presenting some characteristics of sound as vibrations.

show abstract

Section: The Present Studymentioning

confidence: 94%

Vibrotactile Display of Music on the Human Back

Branje

Maksimouski²,

Karam

et al. 2010

2010 Third International Conference on Advances in Computer-Human Interactions

Self Cite

View full text Add to dashboard Cite

show abstract

“…The research from Branje's (2010) directly contributes to the development of the Model Human Cochlea (MHC) [55]. The MHC is a sensory substitution technique based on the human cochlea that presents music as multiple discrete vibration channels on a haptic display [55,66,67]. The goal of the MHC is to translate the emotional content of sound from music and film onto a haptic display to enhance the user's entertainment experience [67].…”

Section: Vibration As Sensory Substitutionmentioning

confidence: 99%

Designing, Implementing and Evaluating a Cross-Modal Sensory Substitution System for the Effective Communication of the Emotional and Informative Aspects of Music

Pouris¹

2023

Preprint

View full text Add to dashboard Cite

Visualizations have existed for millennia as ways to communicate information. Visualizations are ubiquitous tools used every day to help navigate cities and aid in learning complex tasks. Tasks are made simpler when applying various visualization methods to large data sets to discover trends that are otherwise difficult to notice. More recently, music visualization systems have been created to convey music in the visual domain; however, they are not based on any psychological model of auditory and visual equivalents. This thesis discusses a music visualization system called MusicViz, which facilitates in the visual communication of the informative and entertainment aspect of music based on psychologically justified translation principals. MusicViz is combined with a vibro-feedback chair called the Emoti-Chair, which translates auditory music to vibrations along the user’s back. The combined system is coined VITA (Visually Immersive and Tactile Animation). A usability evaluation of the VITA showed it is an enjoyable experience.

show abstract

“…Sensory supplements augment the perceptual information of the user with additional, possibly redundant, information using another sense. Cross-modal displays, where information that is designed to be presented to one sensory modality (i.e., audio, visual, and tactile) is displayed using an alternative modality, is the area of research in human-computer interaction that is involved in sensory substitution and supplementation (Karam & Fels, 2008).…”

Section: Sensory Substitution or Sensory Supplementationmentioning

confidence: 99%

“…Sensory substitution systems designed for D/HH users include tactile displays such as the "emotichair" (Karam & Fels, 2008) and the "artificial ear" (Richardson & Frost, 1977). Other examples include signalling systems which are designed to visually present environmental sounds.…”

Section: Sensory Substitution or Sensory Supplementationmentioning

confidence: 99%

“…When used to substitute one modality for the other, to support users who are blind or deaf, there is a need to identify what characteristics of the input modality are necessary to present using the alternative display (Karam & Fels, 2008) and what characteristics will be lost / missing as a result of these choices. For these reasons supplementing a person's existing sensory capabilities is always preferred over sensory substitution.…”

Section: Sensory Substitution or Sensory Supplementationmentioning

confidence: 99%

See 1 more Smart Citation

Making the invisible visible: visualization of music and lyrics for deaf and hard of hearing audiences

Fourney¹

2021

Preprint

View full text Add to dashboard Cite

Music is a major art form present in all cultures. It is nearly ubiquitous in Western culture. As a shared social experience, music is a shared knowledge space. Some music is so well recognized that people may refer to it by name (e.g., Beethoven’s 5th symphony, the Star Wars theme). Signing Deaf communities have indigenous forms of lyrical music. While these may not be melodic in the Western expectation, they still conform to the idea of music as metrical lyrics accompanied by instrumentation. Music is a system made up of several elements and attributes containing discrete information chunks that can be perceived by a human listener. Music communicates complex emotional signals and, in some cultures, has specific conventions designed to manipulate emotions. However, deaf, deafened, and hard of hearing people are unable to fully access the richness of the aural musical experience. This dissertation explores ways to make information that is presented in an auditory modality accessible to those who cannot fully perceive it due to hearing loss or environmental barriers; making the invisible visible. A multi-disciplinary, multi-method project, this research examines the design of visualizations intended to present different kinds of otherwise invisible music information in a visible, entertaining form. Ninety-four people, 66 women and 28 men, who identified themselves as Hard of Hearing (46), Deaf (38), Cochlear Implantees (8), or Deafened (2), participated in one of four experiments. Eight of the 94 participated in a fifth experiment. Research questions included determining the fundamental properties of music that visualizations must present, determining what emotional responses users might have to visualizations of music information, and determining whether a music experience similar to that of hearing audiences can be experienced via visualizations of music information. Among several findings, this project learned that many deaf, deafened and hard of hearing people are active consumers of music; lyrics represent a key piece of music information hard of hearing people require of any visualization; different visualizations might be better suited to different musical genres or emotional moods; and the set of music visualizations tested convey some types of emotional information but do not obviously communicate the emotional component of the music.

show abstract

Designing a model human cochlea: issues and challenges in crossmodal audio-haptic displays

Cited by 11 publications

References 0 publications

Vibrotactile Display of Music on the Human Back

Vibrotactile Display of Music on the Human Back

Designing, Implementing and Evaluating a Cross-Modal Sensory Substitution System for the Effective Communication of the Emotional and Informative Aspects of Music

Making the invisible visible: visualization of music and lyrics for deaf and hard of hearing audiences

Contact Info

Product

Resources

About