Sound Coding Color to Improve Artwork Appreciation by People with Visual Impairments

Cho, Jun-Dong; Jeong, Jaeho; Kim, Ji Hye; Lee, Hoonsuk

doi:10.3390/electronics9111981

Cited by 14 publications

(29 citation statements)

References 24 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, the code decomposed a specific color into a hue and a set of color dimensions (saturated, light, and dark) for each hue. The VIVALDI (see Cho et al, 2020 ) color code was used to facilitate hue identification (for example, red and orange are represented by string instruments; yellow and green by brass instruments; blue and purple by percussion instruments). To express the saturated, light, and dark color dimensions, VIVALDI uses a different set of pitches for each dimension and fragments of Vivaldi's Four Seasons (Spring, Autumn, and Summer, respectively).…”

Section: Crossmodal Harmonymentioning

confidence: 99%

Crossmodal Harmony: Looking for the Meaning of Harmony Beyond Hearing

Spence

Stefano

2022

i-Perception

View full text Add to dashboard Cite

The notion of harmony was first developed in the context of metaphysics before being applied to the domain of music. However, in recent centuries, the term has often been used to describe especially pleasing combinations of colors by those working in the visual arts too. Similarly, the harmonization of flavors is nowadays often invoked as one of the guiding principles underpinning the deliberate pairing of food and drink. However, beyond the various uses of the term to describe and construct pleasurable unisensory perceptual experiences, it has also been suggested that music and painting may be combined harmoniously (e.g., see the literature on “color music”). Furthermore, those working in the area of “sonic seasoning” sometimes describe certain sonic compositions as harmonizing crossmodally with specific flavor sensations. In this review, we take a critical look at the putative meaning(s) of the term “harmony” when used in a crossmodal, or multisensory, context. Furthermore, we address the question of whether the term's use outside of a strictly unimodal auditory context should be considered literally or merely metaphorically (i.e., as a shorthand to describe those combinations of sensory stimuli that, for whatever reason, appear to go well together, and hence which can be processed especially fluently).

show abstract

Section: Crossmodal Harmonymentioning

confidence: 99%

Crossmodal Harmony: Looking for the Meaning of Harmony Beyond Hearing

Spence

Stefano

2022

i-Perception

View full text Add to dashboard Cite

show abstract

“…Regarding aesthetics in sounds for color substitution, Cho et al [6] investigated possibilities for creating beautiful sounds for representing colors by replicating the three main characteristics of color: hue, chroma, and value, by matching them to three features of sound: timbre, intensity, and pitch. Then, two sets of musical sounds for expressing colors were designed and tested: VIVALDI and CLASSIC.…”

Section: Sound-color Cross-modalitymentioning

confidence: 99%

“…The work is the continuation of a series of works whose aim is to develop a multi-modal system and algorithms that aid visually impaired people in experiencing pieces of art. In previous works, both a color-temperature mapping algorithm and a sound-color coding system were designed, implemented into physical prototypes, and tested [5][6][7]. This work introduces a method for finding the best way to turn those unisensory systems into an improved sound-temperature multi-sensory system for conveying colors.…”

Section: Introductionmentioning

confidence: 99%

Multi-Sensory Color Expression with Sound and Temperature in Visual Arts Appreciation for People with Visual Impairment

Bartolomé

Cho

2021

Electronics

Self Cite

View full text Add to dashboard Cite

For years the HCI community’s research has been focused on the hearing and sight senses. However, in recent times, there has been an increased interest in using other types of senses, such as smell or touch. Moreover, this has been accompanied with growing research related to sensory substitution techniques and multi-sensory systems. Similarly, contemporary art has also been influenced by this trend and the number of artists interested in creating novel multi-sensory works of art has increased substantially. As a result, the opportunities for visually impaired people to experience artworks in different ways are also expanding. In spite of all this, the research focusing on multimodal systems for experiencing visual arts is not large and user tests comparing different modalities and senses, particularly in the field of art, are insufficient. This paper attempts to design a multi-sensory mapping to convey color to visually impaired people employing musical sounds and temperature cues. Through user tests and surveys with a total of 18 participants, we show that this multi-sensory system is properly designed to allow the user to distinguish and experience a total of 24 colors. The tests consist of several semantic correlational adjective-based surveys for comparing the different modalities to find out the best way to express colors through musical sounds and temperature cues based on previously well-established sound-color and temperature-color coding algorithms. In addition, the resulting final algorithm is also tested with 12 more users.

show abstract

“…The auditory codings might be used for conveying color information independently, or in conjunction with tactile sense. Cho et al [6] have studied these relationships and proposed two sound coding color melodies. They considered the tone, intensity, and pitch of melody sounds extracted from classic music to express the brightness and saturation of colors.…”

Section: Introductionmentioning

confidence: 99%

ColorWatch: Color Perceptual Spatial Tactile Interface for People with Visual Impairments

2021

Self Cite

View full text Add to dashboard Cite

Tactile perception enables people with visual impairments (PVI) to engage with artworks and real-life objects at a deeper abstraction level. The development of tactile and multi-sensory assistive technologies has expanded their opportunities to appreciate visual arts. We have developed a tactile interface based on the proposed concept design under considerations of PVI tactile actuation, color perception, and learnability. The proposed interface automatically translates reference colors into spatial tactile patterns. A range of achromatic colors and six prominent basic colors with three levels of chroma and values are considered for the cross-modular association. In addition, an analog tactile color watch design has been proposed. This scheme enables PVI to explore artwork or real-life object color by identifying the reference colors through a color sensor and translating them to the tactile interface. The color identification tests using this scheme on the developed prototype exhibit good recognition accuracy. The workload assessment and usability evaluation for PVI demonstrate promising results. This suggest that the proposed scheme is appropriate for tactile color exploration.

show abstract

Sound Coding Color to Improve Artwork Appreciation by People with Visual Impairments

Cited by 14 publications

References 24 publications

Crossmodal Harmony: Looking for the Meaning of Harmony Beyond Hearing

Crossmodal Harmony: Looking for the Meaning of Harmony Beyond Hearing

Multi-Sensory Color Expression with Sound and Temperature in Visual Arts Appreciation for People with Visual Impairment

ColorWatch: Color Perceptual Spatial Tactile Interface for People with Visual Impairments

Contact Info

Product

Resources

About