Electromyography-Based Decoding of Dexterous, In-Hand Manipulation of Objects: Comparing Task Execution in Real World and Virtual Reality

Abstract

The increased use of Virtual and Augmented Reality based systems necessitates the development of more intuitive and unobtrusive means of interfacing. Over the last years, Electromyography (EMG) based interfaces have been employed for interaction with robotic and computer applications, but no studies have been carried out to investigate the continuous decoding of the effects of human motion (e.g., manipulated object behavior) in simulated and virtual environments. In this work, we compare the object motion decoding accuracy of an EMG based learning framework for two different dexterous manipulation scenarios: i) for simulated objects handled by a teleoperated model of a hand within a virtual environment and ii) for real, everyday life objects manipulated by the human hand. To do that, we utilize EMG activations from 16 muscle sites (9 on the hand and 7 on the forearm). The object motion decoding is formulated as a regression problem using the Random Forests methodology. A 5-fold cross validation procedure is used for model assessment purposes and the feature variable importance values are calculated for each model. The decoding accuracy for the real world is considerably higher than the virtual world. Each of the objects examined had a single manipulation motion that offered the highest estimation accuracy across both worlds. This study also shows that it is feasible to decode the object motions using just the myoelectric activations of the muscles of the forearm and the hand. This is particularly surprising since simulations lacked haptic feedback and the ability to account for other dynamic phenomena like friction and contact rolling.

Intelligent Human–Computer Interaction: Combined Wrist and Forearm Myoelectric Signals for Handwriting Recognition

“…Indeed, the literature provides a variety of software packages that facilitate the training and validation of EMG-based human-machine interfaces able to decode the human intent of motion within a given set of movements [1,2]. Such interfaces find application in prosthetic control of upper and lower bionic limbs [3,4], but also for the realization of intelligent human-computer interactions in virtual and augmented reality, and for biometric identification [5][6][7][8]. Myoelectric interfaces can also find application in modern scenarios since they can be used to decode handwritten characters or digits [9,10], supporting the development of immersive rehabilitation protocols with a consistent involvement of the cognitive centres of the brain [11].…”

ARMIA: A Sensorized Arm Wearable for Motor Rehabilitation

“…This technique has been widely used to evaluate factors such as muscle fatigue [ 16 ] or movement coordination [ 17 ]. sEMG can also be used as input control for virtual environments by processing the activity of muscle contractions [ 18 ]. sEMG provides several advantages compared to other electrophysiological measurements, including non-invasiveness, real-time monitoring and on-site application with relatively affordable equipment.…”

A Systematic Review of Sensor Fusion Methods Using Peripheral Bio-Signals for Human Intention Decoding

“…Such interfaces can be developed employing different sensing modalities to measure the muscle activity or movement. Some of the most common modalities are, but not limited to: electromyography (EMG) [3][4][5][6][7][8][9][10], ultrasonography (US) [11][12][13][14][15], mechanomyography (MMG) [16][17][18][19], and near-infrared spectroscopy (NIRS) [20,21].…”