Interactive feedbacks with visualisation and sonification for technology-enhanced learning for music performance

Ng, Kia

doi:10.1145/1456536.1456599

Cited by 8 publications

(7 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motion capture systems for use in violin performance have utilized online (realtime) feedback of results through Hodgson plots that show the trajectory of the bow in relation to the bridge and the strings (Schoonderwaldt and Wanderley 2007). Ng (2008) used the concept of a "3D augmented mirror" to provide feedback to the performers so that both the performer and the performing environment could be visualized. Both systems (Schoonderwaldt and Wanderley 2007;Ng 2008) traced the trajectory of bow movements in real time so the performer could keep track of their motion patterns during performance.…”

Section: Data Visualizationmentioning

confidence: 99%

Une interface pour utilisateur de caméra 3D employée pour enregistrer les angles des poignets lors de prestations au piano

MacRitchie

Baylis

2016

Les Cahiers de la Société québécoise de recherche en musique

View full text Add to dashboard Cite

Injuries are common over a performing musician’s career and wrist injuries are the most frequent site of pain for pianists. Although general recommendations insist on keeping wrists in a “neutral” position to avoid injury, this is rarely done in practice. Recent advances in motion capture technology may aid in raising students’ awareness of the propensity to use wrist positions outside of the recommended “neutral.” These technologies may be used to measure precise wrist positions in piano playing in order to set specific thresholds for avoiding injury. This paper discusses various advantages and limitations of motion capture technologies, including data visualization and usage within the music instrument pedagogy framework in order to define a set of requirements for an accessible motion-tracking system. A prototype of a dedicated image-processing-based system with a graphical user interface that meets these requirements is described. This system uses passive coloured markers and a standard 3D camera, encouraging use outside the traditional laboratory environment. Simple camera calibration options and basic hand tracking from aerial view images allow monitoring of wrist flexion/extension over short video recordings. Measurements are compared to flexion/extension thresholds recommended for typists to prevent carpal tunnel pressure, and moments of approaching or exceeding these thresholds are flagged to the user both in real time and in post-performance. Potential applications include monitoring the practice of short technical passages without restriction of instrument or location.

show abstract

Section: Data Visualizationmentioning

confidence: 99%

Une interface pour utilisateur de caméra 3D employée pour enregistrer les angles des poignets lors de prestations au piano

MacRitchie

Baylis

2016

Les Cahiers de la Société québécoise de recherche en musique

View full text Add to dashboard Cite

show abstract

“…Multimedia and e-learning conferences also have music training and education works presented (Percival, Wang, and Tzanetakis 2007;Nakano, Goto, and Hiraga 2007;Ng 2008;Ng, Nesi, and McKenzie 2008). Robine, Percival, and LaGrange (2007) and Schoonderwaldt, Askenfelt, and Hansen (2005) address several other performance aspects, such as dynamics, rhythm, and articulation.…”

Section: Past Workmentioning

confidence: 99%

“…Whereas we target any instrument, these works target a limited range of instruments: Ng (2008) and Ng, Nesi, and McKenzie (2008) designed their system for strings, and CantOvation (2008); Chaumet Software (2010); Nakano, Goto, and Hiraga (2007); and VoiceVista (2007) for vocal training. Hoppe, Sadakata, and Desain (2006) presents a review of four different software tools for vocal training with realtime feedback.…”

Section: Past Workmentioning

confidence: 99%

InTune: A System to Support an Instrumentalist's Visualization of Intonation

Lim

Raphael

2010

Computer Music Journal

View full text Add to dashboard Cite

One of our most beloved music teachers emphasized the importance of "facing the music," by which he meant listening to recordings of our playing. As with the first hearing of one's voice on a recording, many of us were both surprised by and suspicious of this external perspective. Despite sometimes revealing more than we are ready to hear, this exercise has the long-term effect of helping us to "hear ourselves as others hear us." Thus armed, we initiate practice habits that, perhaps over many years, move our music-making toward a state we would admire hearing from another player.

show abstract

“…A notable example is i-Maestro [14], which uses motion capture and playback to help violinists visualize and reflect on their playing.…”

Section: Music Learning Interfacesmentioning

confidence: 99%

MirrorFugue

Xiao

Ishii

2010

Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction

View full text Add to dashboard Cite

Playing a musical instrument involves a complex set of continuous gestures, both to play the notes and to convey expression. To learn an instrument, a student must learn not only the music itself but also how to perform these bodily gestures. We present MirrorFugue, a set of three interfaces on a piano keyboard designed to visualize hand gesture of a remote collaborator. Based their spatial configurations, we call our interfaces Shadow, Reflection, and Organ. We describe the configurations and detail studies of our designs on synchronous, remote collaboration, focusing specifically on remote lessons for beginners. Based on our evaluations, we conclude that displaying the to-scale hand gestures of a teacher at the locus of interaction can improve remote piano learning for novices.

show abstract

Interactive feedbacks with visualisation and sonification for technology-enhanced learning for music performance

Cited by 8 publications

References 1 publication

Une interface pour utilisateur de caméra 3D employée pour enregistrer les angles des poignets lors de prestations au piano

Une interface pour utilisateur de caméra 3D employée pour enregistrer les angles des poignets lors de prestations au piano

InTune: A System to Support an Instrumentalist's Visualization of Intonation

MirrorFugue

Contact Info

Product

Resources

About