Towards Developing a Virtual Guitar Instructor through Biometrics Informed Human-Computer Interaction

Abstract

Fig. 1. Capturing guitar performance information via our developed multimodal system, operating across two networked computers (accommodating two Myo wearable sensors, a microphone and a video camera).Within the last few years, wearable sensor technologies have allowed us to access novel biometrics that give us the ability to connect musical gesture to computing systems. Doing this affords us to study how we perform musically and understand the process at data level. However, biometric information is complex and cannot be directly mapped to digital systems. In this work, we study how guitar performance techniques can be captured/analysed towards developing an AI which can provide real-time feedback to guitar students.We do this by performing musical exercises on the guitar whilst acquiring and processing biometric (plus audiovisual) information during their performance. Our results show: there are notable differences within biometrics when playing a guitar scale in two different ways (legato and staccato) and this outcome can be used to motivate our intention to build an AI guitar tutor.CCS Concepts: • Human-centered computing → Gestural input.