Combined Auditory and Vibrotactile Feedback for Human–Machine-Interface Control

Abstract

The purpose of this study was to determine the effect of the addition of binary vibrotactile stimulation to continuous auditory feedback (vowel synthesis) for human-machine-interface (HMI) control. Sixteen healthy participants controlled facial surface electromyography to achieve two-dimensional (2D) targets (vowels). Eight participants used only real-time auditory feedback to locate targets whereas the other eight participants were additionally alerted to having achieved targets with confirmatory vibrotactile stimulation at the index finger. All participants trained using their assigned feedback modality (auditory alone or combined auditory and vibrotactile) over three sessions on three days and completed a fourth session on the third day using novel targets to assess generalization. Analyses of variance performed on the 1) percentage of targets reached and 2) percentage of trial time at the target revealed a main effect for feedback modality: participants using combined auditory and vibrotactile feedback performed significantly better than those using auditory feedback alone. No effect was found for session or the interaction of feedback modality and session, indicating a successful generalization to novel targets but lack of improvement over training sessions. Future research is necessary to determine the cognitive cost associated with combined auditory and vibrotactile feedback during HMI control.

Discrete Versus Continuous Mapping of Facial Electromyography for Human–Machine Interface Control: Performance and Training Effects

“…sEMG systems do not require any particular lighting, and the user need not be directly in front of the computer screen, as in many eye and head tracking systems. The systems described in this paper use visual feedback and therefore also require some intact vision, but alternate systems could easily be adapted for users with visual impairments [29]. Further, reports have suggested that eye-tracking use is fatiguing to eye muscles [7]; our qualitative data show that these sEMG systems are not fatiguing during 1–1.5 hour sessions, but further research will examine the effects of longer usage sessions.…”

Effect of Age on Human–Computer Interface Control Via Neck Electromyography

“…Recent studies have shown promise for using residual surface electromyography (sEMG) from the neck and face to interact with software. Several groups have been exploring neck and face sEMG as an input modality that allows selection of targets in 2D space such as moving a computer cursor on a screen in which holding a static target activation is needed to make a selection in the interface ( Larson et al , 2013 ; Perez-Maldonado et al , 2010 ; Thorp et al , 2014 ; Williams and Kirsch, 2008 ). The aim is to provide a communication interface for severely paralyzed individuals, including those with a history of stroke or C1–C3 spinal cord injuries, who may lack primary communication modalities, including speech and manual control (e.g.…”

Learning to modulate the partial powers of a single sEMG power spectrum through a novel human–computer interface

“…Our device is one of many EMG-driven devices for control of external machines that all require subjects to manipulate EMG in different ways. The most commonly known application of EMG occurs in control of powered prostheses (Oskoei & Hu, 2007;Roche, Rehbaum, Farina, & Aszmann, 2014;Scheme & Englehart, 2011;Zecca, Micera, Carrozza, & Dario, 2002); however, other interfaces also exist that utilize EMG in slightly different ways (Chin, Barreto, Cremades, & Adjouadi, 2008;Cler & Stepp, 2015;Hands, Larson, & Stepp, 2014;Larson, Terry, & Stepp, 2012;Song, Jung, Lee, & Bien, 2009;Thorp, Larson, & Stepp, 2014). For example, Cler and Stepp (2015) have developed a novel spelling device in which users navigate and select keys on a keyboard based on sEMG recorded at five different facial sites.…”

“…For example, Cler and Stepp (2015) have developed a novel spelling device in which users navigate and select keys on a keyboard based on sEMG recorded at five different facial sites. EMG has also been used to control powered wheelchairs (Oonishi, Oh, & Hori, 2010;Song et al, 2009) and in cursor control paradigms, where positions were determined by the contractions recorded on the neck and/or face , Thorp et al, 2014, Williams & Kirsch, 2008.…”