Implementation Considerations in Enabling Visually Impaired Musicians to Read Sheet Music Using a Tablet

Abstract

In this paper, we present the issues to be addressed and the practical solutions to these issues in a mobile application framework for reading and displaying musical scores enhanced to assist the visually impaired in reading and perform the pieces. This framework, currently operating on MusicXML input files, provides the structures and methods for developers to adapt for other music encoding file formats. It also provides the flexible user-settable colors and enlargement parameters to meet the needs of users with various visual impairments. The development challenges fall into three categories: Variable visual impairment driven requirements; Musical notation complexity, and screen realestate limitations of a 10-inch tablet. The framework's practical solutions to each of these challenges are presented and contrasted with traditional solutions and competing solutions.

The challenge of reading music notation for pianists with low vision: An exploratory qualitative study using a head-mounted display

“…Essentially, pitch is represented by the vertical position of notes on the staff, and time is represented by the horizontal position of notes. However, it is equally necessary to distinguish these elements in order to identify the relative length of a note: for example, a full or empty head, the form of the stem (whether it is present or not, and if there are flags or beams), or the presence of a dot (Gudmundsdottir, 2010;Housley, Lynch, Ramnath, Rogers, & Ramanathan, 2013;Read, 1979).…”

“…Accessing this graphical presentation of music requires both detail vision and global vision (Housley et al, 2013). A musician therefore requires good acuity to identify both the pitch and the length of notes (head, stem, flag, beam, dot).…”

“…It therefore involves three specific challenges: (1) reading the notes at an intermediate reading distance (50 cm to 70 cm), (2) having the hands free to play the music, and in the case of playing piano (3) occasionally glancing down at the hands/keyboard. When the musician has a VI, magnification (with different means) and memorization are recommended to facilitate the reading of music; however, they increase cognitive load and task complexity (Bouabid, 2013;Coates, 2012;Housley et al, 2013;Jacko, Choi, Carballo, Charlson, & Moore, 2015), in addition to dealing with challenges such as glare caused by spotlights in concert halls, or poor ligihting in a rehearsal rooms.…”

“…Optical or electronic aids can be used to magnify the score (Coates, 2012;Labbett, 1995;Whittaker et al, 2016). However, since most of these aids are designed to facilitate the reading of text, they present certain problems when reading music (Coates, 2012;Housley et al, 2013). A table-mounted video magnifier provides access to the details in the notation, but the musician needs to stop playing to move the music on the CCTV platform (Clark & Murphy, 1998).…”

Development of a Music Score Editor based on MusicXML

“…However very little published work is available on the development of music score editors. In [9], the authors have developed a MusicXML based music score reader for display of score on a Tablet PC for the visually impaired Musicians. [10] also describes sheet music display on an android Tablet PC using MusicXML.…”