Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates

Abstract

Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain–machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

Cognitive Enhancement: Methods, Ethics, Regulatory Challenges

“…Development is rapid, both on the hardware side, where multielectrode recordings from more than 300 electrodes permanently implanted in the brain are currently state-of-the art, and on the software side, with computers learning to interpret the signals and commands (Nicolelis et al 2003;Shenoy et al 2003;Carmena et al 2003). Early experiments on humans have shown that it is possible for profoundly paralyzed patients to control a computer cursor using just a single electrode (Kennedy and Bakay 1998) implanted in the brain, and experiments by Parag Patil and colleagues have demonstrated that the kind of multielectrode recording devices used in monkeys would most likely also function in humans (Peterman et al 2004;Patil et al 2004).…”

Non-obstructing 3D depth cues influence reach-to-grasp kinematics

“…In the last 10 years in particular, profound advancements have been made in the brain-computer/machine interface field such that primates, healthy humans, and even humans suffering from paralysis are able to make reach-to-grasp movements using a neural prosthetic limb prosthesis (Carmena et al 2003;Nair 2013;Bensmaia & Miller 2014). Upon taking into account the present findings, the computational brain-to-computer decode algorithms employed by these devices to decode neural signals can (and should) be improved.…”

Techniques for extracting single-trial activity patterns from large-scale neural recordings

“…An implicit form of dimensionality reduction is often performed in the context of neural prosthetic systems, when the trajectory of the arm is 'decoded' from simultaneously-recorded neurons [62][63][64]. High (~100) dimensional neural data is collapsed into a low (e.g., 3) dimensional arm trajectory estimate.…”

“…The decoded trajectory is thus a concise 'explanation' or summary of the high-dimensional neural data. Decoding techniques include linear filters [63,64], the population vector [62,65,66], and recursive Bayesian decoding using state-space models [67][68][69]. Most of these approaches attempt to infer something that can be directly observed/inferred on most trials (e.g., actual or expected arm trajectory), yet in some ways this is an advantage, as it allows evaluation of the performance of different decoding techniques.…”