Proportional and Simultaneous Real-Time Control of the Full Human Hand From High-Density Electromyography

Abstract

<p>Surface electromyography (sEMG) is a non-invasive technique that measures the electrical activity generated by the muscles using sensors placed on the skin. It has been widely used in the field of prosthetics and other assistive systems because of the physiological connection between muscle electrical activity and movement dynamics. However, most existing sEMG-based decoding algorithms show a limited number of detectable degrees of freedom that can be proportionally and simultaneously controlled in real-time, which limits the use of EMG in a wide range of applications, including prosthetics and other consumer-level applications (e.g., human/machine interfacing). In this work, we surpass the current state of the art by developing a new deep learning method that can decode and map the electrophysiological activity of the forearm muscles into proportional and simultaneous control of > 20 degrees of freedom of the human hand with real-time resolution and with latency within the neuromuscular delays (< 50 ms). We recorded the kinematics of the human hand during grasping, pinching, individual digit movements and three gestures at slow (0.5 Hz) and fast (0.75 Hz) movement speeds in healthy participants.</p> <p>We demonstrate that our neural network can predict the kinematics of the hand in real-time at a constant 32 predictions per second. To achieve this, we employed transfer learning and created a prediction smoothing algorithm for the output of the neural network that reconstructed the full geometry of the hand in three-dimensional Cartesian space in real-time. Our results demonstrate that high-density EMG signals from the forearm muscles contain almost all the information that is needed to predict the kinematics of the human hand. The proposed method has the capability of predicting the full kinematics of the human hand in an unprecedented way and with real-time resolution with immediate translational impact in subjects with motor impairments. </p>

Identification of Spared and Proportionally Controllable Hand Motor Dimensions in Motor Complete Spinal Cord Injuries Using Latent Manifold Analysis

“…Each movement was displayed through a virtual hand. Using the EMG and synchronized hand kinematics data, we trained our previously published AI model [11] capable of real-time kinematics prediction for each participant individually to assess whether proportional control can be used in individuals with SCI and if so for which movements.…”

“…Using the EMG data collected, which were synced with the digitally displayed hand kinematics, we trained our previously published model [11], which is capable of accurately and continuously predicting hand kinematics in real-time for each individual participant. The model was implemented in Python using PyTorch (version 2.1.0+cu121) and PyTorch-Lightning (version 2.1.0).…”

“…The last part of the model is a multilayer perceptron that maps the high-dimensional latent space into the desired 3D coordinates. A detailed explanation can be found in Sîmpetru et al [11]. D. Using UMAP, a non-linear dimensionality reduction technique that preserves the local and global information, we project the latent space vectors into human understandable 2D.…”

“…The potential of having an AI system that automatically learns the EMG activation patterns generating specific movements has numerous applications for the recovery of the hand function, considering the large number of motor dimensions embedded in the hand. For this, we adapted our previously published deep learning model capable of realtime, accurate and proportional hand kinematics prediction for healthy participants [11] and tested it on 13 uninjured (control group) and 8 individuals suffering from a motorcomplete SCI (SCI group).…”

High-Performance Hand Kinematics Estimation Requires Full Bandwidth High-Density EMG Recordings

“…1). We have recently developed a neural network that is capable of predicting the full kinematics of the human hand at multiple degrees of freedom [3] with simultaneous and proportional real-time control [4]. Here, we aimed to investigate the impact of the pre-processing filters and ablation methods on the accuracy of the neural network in predicting hand movements based on the processed EMG signals.…”