‘Don’t follow them, look at me!’: Contemplating a haptic digital prototype to bridge the conductor and visually impaired performer

Abstract

This paper reports on an exploratory research-and-development project concerning a device for conveying a conductor's gestures wirelessly to a visually impaired (blind or partially sighted) performer as a haptic signal. The research team developed this device from January to July 2017 under a University College London (UCL) Institute of Education "seed-corn" grant. As a platform for its development, they firstly observed and analysed video footage of conductors at the Royal Academy of Music, London using Elan software to create a gestural model. Subsequently, through gaining blind end-users' feedback on the device, as well as an experiment to compare their timing using either (i) a two-dimension haptic signal or (ii) a metronomic pulsation, it is suggested that the development of technologies for this purpose should focus on the meaning the conductor intends to convey coupled with haptic signals blind end-users themselves deem suitable, rather than adopting a "sighted perspective" in attempting faithful transference of two-dimensional captures of arm movements from one medium to another. Reasons for this assertion are explored. THE ENSEMBLE EXPERIENCES OF THE VISUALLY IMPAIREDVisually impaired musicians participate in some music ensembles with limited or no reference to the conductor's gestures by contrast with their sighted counterparts. These musicians navigate the experience variously depending on particular sight conditions and personal approaches; these are not without challenges. I've done those "Come and sing" events, so "Come and sing Mozart's requiem, come and sing Messiah" just for fun. …They [the sighted singers around me] are obviously just reading their music and not watching the conductor. Sometimes conductors say "Don't listen to them, go in time with me, look at me" and I think "Maybe I should just go home then" (laughs). (Felicity, keyboard player and singer, blind) Felicity 1 is a visually impaired 2 singer who performs in choirs alongside sighted musicians. Her remark above highlights a perennial issue: Integration into many types of conducted musical 1 Pseudonyms have been used in this paper for every respondent. 2 We have used the term "visually impaired" as an umbrella term to denote those with sight health problems or the absence of sight such that they meet the requirements of UK registration by an ophthalmologist. Under this umbrella there may be people who the layperson may describe as "blind" or "partially sighted", and we use those terms too. Other terms in common usage include "sightless", "sight impaired", "low vision", etc. with

Musical Haptic Wearables for Synchronisation of Visually-impaired Performers: a Co-design Approach

“…As a consequence of this, they also said that after a longer training period they could have been more efcient in responding to the musical directions. Notably, this need for a longer time to get familiar with and responsive to the tactile cues parallels the fndings reported in other studies in musical haptics [3,14,32].…”

“…A related strand of research has prioritised accessibility aspects related to visually-impaired musicians, which represents an important aspect for social inclusion of this section of the worldwide population [3,4] visually-impaired musicians in various activities. This includes haptic tools for studio production (e.g., haptic waves, a device that allows cross-modal mapping of digital audio to the haptic domain [28]), systems for interactions between conductors and blind choristers [11], and haptic wearables for integrating visually-impaired musicians in orchestras of sighted musicians (e.g., [2,18]).…”

Digital Technologies in Music Education: The Case of Chinese Students

“…have suggested the use of virtual and augmented reality technologies in music education (SERAFIN et al, 2017). One of the advantages of modern technologies in music education is the possibility of visualization through special software or an interactive whiteboard (BAKER et al, 2019).…”

Methodological Experience in the Teaching-Learning of the English Language for Students with Visual Impairment

“…In the last decade, information and communication technology (ICT) has become an inseparable ally of students' academic activities. It allows them to have access to information independently [30], and intelligent devices [31] as technological support tools [16] play a fundamental role for VISs through the different advantages of applications that allow them to be in direct contact with information in a learning environment.…”