The EyeHarp: A Gaze-Controlled Digital Musical Instrument

Abstract

We present and evaluate the EyeHarp, a new gaze-controlled Digital Musical Instrument, which aims to enable people with severe motor disabilities to learn, perform, and compose music using only their gaze as control mechanism. It consists of (1) a step-sequencer layer, which serves for constructing chords/arpeggios, and (2) a melody layer, for playing melodies and changing the chords/arpeggios. We have conducted a pilot evaluation of the EyeHarp involving 39 participants with no disabilities from both a performer and an audience perspective. In the first case, eight people with normal vision and no motor disability participated in a music-playing session in which both quantitative and qualitative data were collected. In the second case 31 people qualitatively evaluated the EyeHarp in a concert setting consisting of two parts: a solo performance part, and an ensemble (EyeHarp, two guitars, and flute) performance part. The obtained results indicate that, similarly to traditional music instruments, the proposed digital musical instrument has a steep learning curve, and allows to produce expressive performances both from the performer and audience perspective.

Classification of Human Emotions from Electroencephalogram (EEG) Signal using Deep Neural Network

“…In Vamvakousis et al [13], the Amyotrophic Lateral Sclerosis (ALS) patients were expressing their emotions through music in real time. They used (1) and (2) for detecting valence and arousal, respectively.…”

“…There has been a lot of research that investigated neural correlates of emotion in humans [6], [9], [12], [13]. Frontal activity, which is characterized in terms of decreased power in the alpha band, has been consistently found to be associated with emotional states [11].…”

“…Besides EEG applications, it has been widely used for numerous applications in engineering, science, and mathematics. In this study, each EEG signal is decomposed using PSD approach into four distinct frequency ranges: theta (4-8 Hz), alpha (8-13 Hz), beta (13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30), and gamma (30-40 Hz). The PSDs were computed using Python Signal Processing Toolbox (mne), and the average of power over a specific frequency range was calculated to construct a feature using the avgpower function in the toolbox.…”

Accessible Digital Musical Instruments—A Review of Musical Interfaces in Inclusive Music Practice

“…The term "adaptive music" is also used by Graham-Knight and Tzanetakis [30], who define it as the use of digital technologies to allow a person who cannot otherwise play a traditional musical instrument to play music unaided. Moreover, the word "adaptive" is used in a study by Vamvakousis and Ramirez [42], who specifically refer to the notion of "Adaptive Digital Musical Instruments". Among these terms, you also encounter "inclusive music" [7], defined by Samuels as the use of music interfaces aimed at overcoming disabling barriers to music-making faced by people with disabilities [21].…”

Auditory Display Applied to Research in Music and Acoustics

“…There are examples in terms of common interests for both Music Technology and AD. Vamvakousis and Ramirez (2012) showed the EyeHarp, an eye-tracking musical interface for controlling melodic, harmonic and expressive aspects of musical instruments in real time, have similar expressive potentials to a traditional musical instrument. In addition, many AD papers related to music could be found within the scope of International Conferences on Auditory Display (ICADs).…”