Gaussian Process Autoregression for Simultaneous Proportional Multi-Modal Prosthetic Control With Natural Hand Kinematics

Abstract

Matching the dexterity, versatility, and robustness of the human hand is still an unachieved goal in bionics, robotics, and neural engineering. A major limitation for hand prosthetics lies in the challenges of reliably decoding user intention from muscle signals when controlling complex robotic hands. Most of the commercially available prosthetic hands use muscle-related signals to decode a finite number of predefined motions and some offer proportional control of open/close movements of the whole hand. Here, in contrast, we aim to offer users flexible control of individual joints of their artificial hand. We propose a novel framework for decoding neural information that enables a user to independently control 11 joints of the hand in a continuous manner-much like we control our natural hands. Toward this end, we instructed six able-bodied subjects to perform everyday object manipulation tasks combining both dynamic, free movements (e.g., grasping) and isometric force tasks (e.g., squeezing). We recorded the electromyographic and mechanomyographic activities of five extrinsic muscles of the hand in the forearm, while simultaneously monitoring 11 joints of hand and fingers using a sensorized data glove that tracked the joints of the hand. Instead of learning just a direct mapping from current muscle activity to intended hand movement, we formulated a novel autoregressive approach that combines the context of previous hand movements with instantaneous muscle activity to predict future hand movements. Specifically, we evaluated a linear vector autoregressive moving average model with exogenous inputs and a novel Gaussian process ( ) autoregressive framework to learn the continuous mapping from hand joint dynamics and muscle activity to decode intended hand movement. Our approach achieves high levels of performance (RMSE of 8°/s and ). Crucially, we use a small set of sensors that allows us to control a larger set of independently actuated degrees of freedom of a hand. This novel undersensored control is enabled through the combination of nonlinear autoregressive continuous mapping between muscle activity and joint angles. The system evaluates the muscle signals in the context of previous natural hand movements. This enables us to resolve ambiguities in situations, where muscle signals alone cannot determine the correct action as we evaluate the muscle signals in their context of natural hand movements. autoregression is a particularly powerful approach which makes not only a prediction based on the context but also represents the associated uncertainty of its predictions, thus enabling the novel notion of risk-based control in neuroprosthetics. Our results suggest that autoregressive approaches with exogenous inputs lend themselves for natural, intuitive, and continuous control in neurotechnology, with the particular focus on prosthetic restoration of natural limb function, where high dexterity is required for complex movements.

Grammars of action in human behavior and evolution

“…Our analytical approach does not postulate the existence of such "action grammars" a priori, but instead identifies them from raw behavioral data using machine learning techniques, showing that even with the same alphabet of actions qualitatively more complex artefacts can be produced by using measurably more complex action grammars. In addition to the method's broad utility for the behavioral and social sciences, the finding that our automatic identification of action grammars maps to distinct neural correlates offers the potential for novel quantitative approaches to the hierarchical structure of behavior across applications from dexterous prosthetics 50 , to the training of surgeons 51 and human-like AI 52 .…”

Neurocognitive barriers to the embodiment of technology

“…However, limitations in the readout of electromyography signals result in non-intuitive and coarse control (for instance, users may be required to switch between discrete hand grasps). Current developments aim to enable the user to control multiple joints of a bionic hand simultaneously and to do it smoothly (that is, through proportional control of the joints) 2 . Still, the use remains counterintuitive at first and requires training.…”

Myoelectric digit action decoding with multi-label, multi-class classification: an offline analysis

“…In myoelectric control, multi-label classification has been previously used to decode simultaneous wrist and hand motions [4][5][6][7][8][9] . For control of prosthetic digits, however, previous efforts have focused on using multi-output regression to reconstruct position [15][16][17][18]20 , velocity 22,23 or fingertip force trajectories [25][26][27] . One study has previously adopted a similar approach to ours 36 , but with three main differences: firstly, the labels corresponded to digit positions instead of actions; secondly, labels were binary (i.e., digits could be fully open or closed), whereas with our approach actions can take three values (i.e., open, close, or stall); finally, due to using binary outputs, a controller using the approach proposed previously would be limited to extreme digit positions.…”

“…From a technical perspective, the ultimate goal of the myoelectric control field is to approximate this dexterity via simultaneous and independent control of multiple degrees of freedom (DOFs) in a continuous space. To that end, several groups, including us, have used regression-based methods to reconstruct wrist kinematic trajectories [10][11][12][13][14] , finger positions [15][16][17][18][19][20][21] and velocities 22,23 , as well as fingertip forces [24][25][26][27] . Only a few studies have, however, thus far demonstrated the feasibility of real-time prosthetic finger control in amputee users 16,20,21 .…”