Neuronal representation of bimanual arm motor imagery in the motor cortex of a tetraplegia human, a pilot study

Lai, Dongrong; Wan, Zijun; Lin, Jiafan; Li, Pan; Ren, Feixiao; Zhang, Jianmin; Wang, Yueming; Hao, Yaoyao; Xu, Kedi

doi:10.3389/fnins.2023.1133928

Cited by 4 publications

References 45 publications

(109 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

Brain control of bimanual movement enabled by recurrent neural networks

Deo,

Willett,

Avansino

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Brain-computer interfaces have so far focused largely on enabling the control of a single effector, for example a single computer cursor or robotic arm. Restoring multi-effector motion could unlock greater functionality for people with paralysis (e.g., bimanual movement). However, it may prove challenging to decode the simultaneous motion of multiple effectors, as we recently found that a compositional neural code links movements across all limbs and that neural tuning changes nonlinearly during dual-effector motion. Here, we demonstrate the feasibility of high-quality bimanual control of two cursors via neural network (NN) decoders. Through simulations, we show that NNs leverage a neural ‘laterality’ dimension to distinguish between left and right-hand movements as neural tuning to both hands become increasingly correlated. In training recurrent neural networks (RNNs) for two-cursor control, we developed a method that alters the temporal structure of the training data by dilating/compressing it in time and re-ordering it, which we show helps RNNs successfully generalize to the online setting. With this method, we demonstrate that a person with paralysis can control two computer cursors simultaneously. Our results suggest that neural network decoders may be advantageous for multi-effector decoding, provided they are designed to transfer to the online setting.

show abstract

Brain control of bimanual movement enabled by recurrent neural networks

Deo,

Willett,

Avansino

et al. 2024

Sci Rep

View full text Add to dashboard Cite

show abstract

Translating deep learning to neuroprosthetic control

Deo

Willett

Avansino

et al. 2023

Preprint

View full text Add to dashboard Cite

Advances in deep learning have given rise to neural network models of the relationship between movement and brain activity that appear to far outperform prior approaches. Brain-computer interfaces (BCIs) that enable people with paralysis to control external devices, such as robotic arms or computer cursors, might stand to benefit greatly from these advances. We tested recurrent neural networks (RNNs) on a challenging nonlinear BCI problem: decoding continuous bimanual movement of two computer cursors. Surprisingly, we found that although RNNs appeared to perform well in offline settings, they did so by overfitting to the temporal structure of the training data and failed to generalize to real-time neuroprosthetic control. In response, we developed a method that alters the temporal structure of the training data by dilating/compressing it in time and re-ordering it, which we show helps RNNs successfully generalize to the online setting. With this method, we demonstrate that a person with paralysis can control two computer cursors simultaneously, far outperforming standard linear methods. Our results provide evidence that preventing models from overfitting to temporal structure in training data may, in principle, aid in translating deep learning advances to the BCI setting, unlocking improved performance for challenging applications.

show abstract