Brain-computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. To date, a major focus of BCI research has been on restoring gross motor skills, such as reaching and grasping 1-5 or point-and-click typing with a 2D computer cursor 6,7 . However, rapid sequences of highly dexterous behaviors, such as handwriting or touch typing, might enable faster communication rates. Here, we demonstrate an intracortical BCI that decodes attempted handwriting movements from neural activity in motor cortex and translates it to text in real-time, using a novel recurrent neural network decoding approach. With this BCI, our study

Neural activity in monkey motor cortex (M1) and dorsal premotor cortex (PMd) can reflect a chosen movement well before that movement begins. The pattern of neural activity then changes profoundly just before movement onset. We considered the prediction, derived from formal considerations, that the transition from preparation to movement might be accompanied by a large overall change in the neural state that reflects when movement is made rather than which movement is made. Specifically, we examined “components” of the population response: time-varying patterns of activity from which each neuron’s response is approximately composed. Amid the response complexity of individual M1 and PMd neurons, we identified robust response components that were “condition-invariant”: their magnitude and time course were nearly identical regardless of reach direction or path. These condition-invariant response components occupied dimensions orthogonal to those occupied by the “tuned” response components. The largest condition-invariant component was much larger than any of the tuned components; i.e., it explained more of the structure in individual-neuron responses. This condition-invariant response component underwent a rapid change before movement onset. The timing of that change predicted most of the trial-by-trial variance in reaction time. Thus, although individual M1 and PMd neurons essentially always reflected which movement was made, the largest component of the population response reflected movement timing rather than movement type.

No abstract

No abstract

The prevailing view of motor cortex holds that motor cortical neural activity represents muscle or movement parameters. However, recent studies in non-human primates have shown that neural activity does not simply represent muscle or movement parameters; instead, its temporal structure is well-described by a dynamical system where activity during movement evolves lawfully from an initial pre-movement state. In this study, we analyze neuronal ensemble activity in motor cortex in two clinical trial participants diagnosed with Amyotrophic Lateral Sclerosis (ALS). We find that activity in human motor cortex has similar dynamical structure to that of non-human primates, indicating that human motor cortex contains a similar underlying dynamical system for movement generation.Clinical trial registration: NCT00912041.DOI: http://dx.doi.org/10.7554/eLife.07436.001

SummaryStudies in multiple species have revealed the existence of neural signals that lawfully co-vary with different aspects of the decision-making process, including choice, sensory evidence that supports the choice, and reaction time. These signals, often interpreted as the representation of a decision variable (DV), have been identified in several motor preparation circuits and provide insight about mechanisms underlying the decision-making process. However, single-trial dynamics of this process or its representation at the neural population level remain poorly understood. Here, we examine the representation of the DV in simultaneously recorded neural populations of dorsal premotor (PMd) and primary motor (M1) cortices of monkeys performing a random dots direction discrimination task with arm movements as the behavioral report. We show that single-trial DVs covary with stimulus difficulty in both areas but are stronger and appear earlier in PMd compared to M1 when the stimulus duration is fixed and predictable. When temporal uncertainty is introduced by making the stimulus duration variable, single-trial DV dynamics are accelerated across the board and the two areas become largely indistinguishable throughout the entire trial. These effects are not trivially explained by the faster emergence of motor kinematic signals in PMd and M1. All key aspects of the data were replicated by a computational model that relies on progressive recruitment of units with stable choice-related modulation of neural population activity. In contrast with several recent results in rodents, decision signals in PMd and M1 are not carried by short sequences of activity in non-overlapping groups of neurons but are instead distributed across many neurons, which once recruited, represent the decision stably during individual behavioral epochs of the trial.

Objective : To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of classifiers trained to discriminate a comprehensive basis set for speech: 39 English phonemes. We classified neural correlates of spoken-out-loud words in the "hand knob" area of precentral gyrus, which we view as a step towards the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak. Approach : Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode's binned action potential counts or high-frequency local field potential power. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes' onset times..

22When making a categorical decision about a noisy stimulus, it is common to fluctuate between 44 levels of commitment to a choice before reporting a decision. In some instances the fluctuations 45 are sufficiently strong to lead to a "change of mind" (CoM) while deliberating [1][2][3][4][5][6] or even while the 46 reporting action is being executed 7 . Because these within-trial fluctuations are different from trial 47 to trial and not necessarily tied to an external event or stimulus feature, they can only be captured 48 using a moment-to-moment neural readout of the decision state on single trials. 49To obtain this readout, we decoded a decision variable (DV) from neural population activity in 50 PMd and M1 in real time to continuously estimate the decision state while two monkeys performed 51 a motion discrimination task 8,9 (Fig. 1a, see Methods). The DV was estimated by applying a linear 52 decoder, trained on data from a previous experimental session, to spiking data (from 96 to 192 53 electrodes) from the preceding 50 ms, updated every 10 ms throughout each trial ( Fig. 1b, see 54 Methods). The sign of the DV indicated which choice was predicted by the decoder, which allowed 55 us to calculate the decoder's prediction accuracy. The DV magnitude reflected the confidence of 56 the model's prediction in units of log-odds for one vs. the other decision (see Methods). Note that 57 the decision variable as defined here encompasses all choice predictive signals that can be decoded 58 from neural activity 10 , including but not limited to moment-to-moment value of accumulated 59 evidence as posited in classical sequential sampling models. 60We have previously demonstrated with offline analysis that this decision variable (DV) can predict 61 choices on single trials up to seconds before initiation of the operant response, and that the 62 accuracy of these predictions increases on average throughout the course of the trial 10 . 63Here, we employed closed-loop, neurally-contingent control over stimulus timing to directly probe 64 the relationship of within-trial DV fluctuations to behaviorally meaningful decision states. For the 65 4 first time, we quantified the behavioral effects of previously covert DV variations (i) as a function 66 of time and for different virtual DV boundaries imposed during the trial, (ii) when large, CoM-like 67 fluctuations were detected during deliberation on noisy visual evidence, and (iii) when 68 subthreshold stimulus pulses were added during the trial. 69Having a nearly instantaneous real-time estimate of the decision state read-out enabled us to 70 terminate the visual stimulus based on the current value (or history) of the DV and validate the 71 behavioral relevance of DV fluctuations using the monkey's behavioral reports following stimulus 72 termination. 73Decisions on perceived stimulus motion can be reliably decoded in real time based on 50 ms 74 of PMd/M1 neural activity 75 76 Two monkeys performed a variable duration variant of the classical random dot motion 77 discriminatio...