Using Interactive Machine Learning to Sonify Visually Impaired Dancers' Movement

Katan, Simon

doi:10.1145/2948910.2948960

Cited by 6 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In an artistic context, sonification can also be used as creative interaction tool to shape movement expression and to produce sound compositions through body movement, which can then be used to stimulate new movement (Landry and Jeon, 2020). Katan (2016) reports on a workshop with three dancers with visual impairment in which the benefits of movement sonification for communication and joint movement were examined. Afterwards, all dancers stated that they enjoyed controlling sound through movement and that they felt encouraged to try new things, however, it did not improve movement comprehension or mutual understanding.…”

Section: Dance Made Audible By Means Of Technologymentioning

confidence: 99%

“…Sonification of dance (and related human movement) has been studied intensively, however, only a small part of this research has been carried out with a non-sighted audience in mind (e.g., Katan, 2016). Investigating this topic systematically could help to gain insights that could establish sonification as a tool for augmenting dance performance.…”

Section: Second Field Of Study: Dance and Movement Sonificationmentioning

confidence: 99%

See 1 more Smart Citation

Dance Is More Than Meets the Eye—How Can Dance Performance Be Made Accessible for a Non-sighted Audience?

Bläsing

Zimmermann

2021

Front. Psychol.

View full text Add to dashboard Cite

Dance is regarded as visual art form by common arts and science perspectives. Definitions of dance as means of communication agree that its message is conveyed by the dancer/choreographer via the human body for the observer, leaving no doubt that dance is performed to be watched. Brain activation elicited by the visual perception of dance has also become a topic of interest in cognitive neuroscience, with regards to action observation in the context of learning, expertise and aesthetics. The view that the aesthetic experience of dance is primarily a visual one is still shared by many artists and cultural institutions, yet there is growing interest in making dance performances accessible for individuals with visual impairment / blindness. Means of supporting the non-visual experience of dance include verbal (audio description), auditive (choreographed body sounds, movement sonification), and haptic (touch tour) techniques, applied for different purposes by artists and researchers, with three main objectives: to strengthen the cultural participation of a non-sighted audience in the cultural and aesthetic experience of dance; to expand the scope of dance as an artistic research laboratory toward novel ways of perceiving what dance can convey; and to inspire new lines of (neuro-cognitive) research beyond watching dance. Reviewing literature from different disciplines and drawing on the personal experience of an inclusive performance of Simon Mayer's “Sons of Sissy,” we argue that a non-exclusively visual approach can be enriching and promising for all three perspectives and conclude by proposing hypotheses for multidisciplinary lines of research.

show abstract

Section: Dance Made Audible By Means Of Technologymentioning

confidence: 99%

Section: Second Field Of Study: Dance and Movement Sonificationmentioning

confidence: 99%

Dance Is More Than Meets the Eye—How Can Dance Performance Be Made Accessible for a Non-sighted Audience?

Bläsing

Zimmermann

2021

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…There has been a recent resurgence in the dancer sonification literature in the past decade (Alborno et al, 2016;de Quay, Skogstad, & Jensenius, 2011;Effenberg, Melzer, Weber, & Zinke, 2005;Ferguson & Beilharz, 2009;Fox & Carlile, 2005;Frid, Elblaus, & Bresin, 2016;Goina & Polotti, 2008;Großhauser, Bläsing, Spieth, & Hermann, 2012;Hermann, Höner, & Ritter, 2005;Kapur, Tzanetakis, Virji-Babul, Wang, & Cook, 2005;Katan, 2016;Lindborg, 2016;Mironcika, Pek, Franse, & Shu, 2016;Naveda & Leman, 2008;Salter, Baalman, & Moody-Grigsby, 2007;R. M. Winters et al, 2012;Yamaguchi & Kadone, 2017).…”

Section: Dancer Sonification Systemsmentioning

confidence: 99%

Interactive sonification strategies for the motion and emotion of dance performances

Landry

Jeon

2020

J Multimodal User Interfaces

View full text Add to dashboard Cite

“…Table 1 reports a list of existing studies on sonification techniques for dance. Many of them, e.g., [1,5,25,26,31], only considered low-level movement features (i.e., at the level of motion capture data, wearable sensors, video, and so on) and mapped them into sound. Studies that proposed sonification models to translate higher-level movement features are less common.…”

Section: Related Workmentioning

confidence: 99%

Does embodied training improve the recognition of mid-level expressive movement qualities sonification?

Niewiadomski

Mancini

Cera

et al. 2018

J Multimodal User Interfaces

View full text Add to dashboard Cite

This research is a part of a broader project exploring how movement qualities can be recognized by means of the auditory channel: can we perceive an expressive full-body movement quality by means of its interactive sonification? The paper presents a sonification framework and an experiment to evaluate if embodied sonic training (i.e., experiencing interactive sonification of your own body movements) increases the recognition of such qualities through the auditory channel only, compared to a non-embodied sonic training condition. We focus on the sonification of two mid-level movement qualities: fragility and lightness. We base our sonification models, described in the first part, on the assumption that specific compounds of spectral features of a sound can contribute to the cross-modal perception of a specific movement quality. The experiment, described in the second part, involved 40 participants divided into two groups (embodied sonic training vs. no training). Participants were asked to report the level of lightness and fragility they perceived in 20 audio stimuli generated using the proposed sonification models. Results show that (1) both expressive qualities were correctly recognized from the audio stimuli, (2) a positive effect of embodied sonic training was observed for fragility but not for lightness. The paper is concluded by the description of the artistic performance that took place in 2017 in Genoa (Italy), in which the outcomes of the presented experiment were exploited. KeywordsSonification • Expressive qualities • Lightness • Fragility • Movement qualities • Embodied training M. Mancini participated to this work while being member of the Casa Paganini-

show abstract

Using Interactive Machine Learning to Sonify Visually Impaired Dancers' Movement

Cited by 6 publications

References 4 publications

Dance Is More Than Meets the Eye—How Can Dance Performance Be Made Accessible for a Non-sighted Audience?

Dance Is More Than Meets the Eye—How Can Dance Performance Be Made Accessible for a Non-sighted Audience?

Interactive sonification strategies for the motion and emotion of dance performances

Does embodied training improve the recognition of mid-level expressive movement qualities sonification?

Contact Info

Product

Resources

About