High-performance brain-to-text communication via imagined handwriting

Willett, Francis R.; Avansino, Donald T.; Hochberg, Leigh R.; Henderson, Jaimie M.; Shenoy, Krishna V.

doi:10.1101/2020.07.01.183384

Cited by 21 publications

(18 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Neuroscientists seek to understand how the brain drives behaviour. Researchers and clinicians often monitor brain activity using implanted electrodes [1][2][3][4] . Modern multielectrode arrays allow the spiking activity of hundreds of neurons to be observed simultaneously 5 .…”

Section: Introductionmentioning

confidence: 99%

Local field potentials reflect cortical population dynamics in a region-specific and frequency-dependent manner

Gallego-Carracedo

Chowdhury

et al. 2021

Preprint

View full text Add to dashboard Cite

The spiking activity of populations of cortical neurons is well described by a small number of population-wide covariance patterns, the "latent dynamics". These latent dynamics are largely driven by the same correlated synaptic currents across the circuit that determine the generation of local field potentials (LFP). Yet, the relationship between latent dynamics and LFPs remains largely unexplored. Here, we characterised this relationship for three different regions of primate sensorimotor cortex during reaching. The correlation between latent dynamics and LFPs was frequency-dependent and varied across regions. However, for any given region, this relationship remained stable across behaviour: in each of primary motor and premotor cortices, the LFP-latent dynamics correlation profile was remarkably similar between movement planning and execution. These robust associations between LFPs and neural population latent dynamics help bridge the wealth of studies reporting neural correlates of behaviour using either type of recordings.

show abstract

Section: Introductionmentioning

confidence: 99%

Local field potentials reflect cortical population dynamics in a region-specific and frequency-dependent manner

Gallego-Carracedo

Chowdhury

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Though not all neural data is decipherable, some of it can be "read" or interpreted. Imagined handwriting, for instance, can be decoded and translated by BCI into quick and accurate texting [44]. Similarly, BCIs that identify and convert covert speech could be used to drive a computerized voice [45] and so externalize what was previously private.…”

Section: Brain Data Privacymentioning

confidence: 99%

Recommendations for Responsible Development and Application of Neurotechnologies

et al. 2021

View full text Add to dashboard Cite

Advancements in novel neurotechnologies, such as brain computer interfaces (BCI) and neuromodulatory devices such as deep brain stimulators (DBS), will have profound implications for society and human rights. While these technologies are improving the diagnosis and treatment of mental and neurological diseases, they can also alter individual agency and estrange those using neurotechnologies from their sense of self, challenging basic notions of what it means to be human. As an international coalition of interdisciplinary scholars and practitioners, we examine these challenges and make recommendations to mitigate negative consequences that could arise from the unregulated development or application of novel neurotechnologies. We explore potential ethical challenges in four key areas: identity and agency, privacy, bias, and enhancement. To address them, we propose (1) democratic and inclusive summits to establish globally-coordinated ethical and societal guidelines for neurotechnology development and application, (2) new measures, including “Neurorights,” for data privacy, security, and consent to empower neurotechnology users’ control over their data, (3) new methods of identifying and preventing bias, and (4) the adoption of public guidelines for safe and equitable distribution of neurotechnological devices.

show abstract

“…Another way of leveraging additional dimensions -or even the full neural state -would be to use a trained RNN to decode parameters of interest (Anumanchipalli et al, 2019;Sussillo et al, 2012;Willett et al, 2020). This is a promising approach, and avoids the need to hand-select features.…”

Section: Population Dynamics In Decodingmentioning

confidence: 99%

Cortical control of virtual self-motion using task-specific subspaces

Schroeder

Perkins

Wang

et al. 2019

Preprint

View full text Add to dashboard Cite

1Brain-machine interfaces (BMIs) for reaching have enjoyed continued performance 2improvements. Yet there remains significant need for locomotor BMIs (e.g., for wheelchair 3 control), which could potentially benefit a much larger patient population. Fewer studies have 4 addressed this need, and the most effective approach remains undetermined. Here, we develop 5 a locomotor BMI based on cortical activity as monkeys cycle a hand-held pedal to progress 6 along a virtual track. Unlike most reach-based BMIs, we did not directly map neural states to 7 commanded velocity or position. Instead, we leveraged features of the neural population 8 response that were robust during rhythmic cycling. These included an overall shift in neural 9 state when moving, and rotational trajectories with direction-specific paths. We used nonlinear 10 means to infer kinematics from these features. Online BMI-control success rates approached 11 those during manual control. Our results illustrate that different use-cases can require very 12 different approaches to guiding a prosthetic via neural activity. 13 14 Brain-machine interfaces (BMIs) interpret neural activity and provide control signals to external 15 devices such as computers and prosthetic limbs. Intracortical BMIs for reach-like tasks have 16 proved successful in primates and human clinical trials 1-8 . More widespread use appears 17imminent. Yet at the same time, there exist non-reach-like movements whose restoration is 18 valuable to patients. For example, many patients could benefit from a BMI that controls 19 locomotion through their environment (e.g., movement of a wheelchair). Recent work has 20 demonstrated that this is feasible 9,10 . While locomotor BMIs can be guided by reach-inspired 21 decoding approaches, other viable strategies exist and remain unexplored. For example, it may 22A decoder that leveraged these dominant features provided excellent online control of virtual 71 locomotion. Success rates and acquisition times were very close to those achieved under manual 72 control. Almost no training or adaptation time was needed; the low-latency and accuracy of the 73 decoder were such that monkeys appeared to barely notice transitions from manual control to 74 BMI control. These results demonstrate the feasibility of BMI locomotion based on rhythmic 75 neural activity. More broadly, they establish that opportunistic decode strategies can work well 76 in non-reach-based scenarios, but that new applications require novel decode approaches that 77 respect the dominant structure of neural activity. 78 79 80 5 Results 81Behavior 82We trained two monkeys (G and E) to rotate a hand-held pedal to move through a virtual 83 environment (Fig. 1). All motion was along a linear track -no steering was necessary. 84Consistent with this, a single pedal was cycled with the right arm only. Our goal when 85 decoding was to reconstruct the virtual motion produced by that single pedal. On each trial, a 86 target appeared in the distance. To acquire that target, monkeys produced virtual ve...

show abstract

High-performance brain-to-text communication via imagined handwriting

Cited by 21 publications

References 39 publications

Local field potentials reflect cortical population dynamics in a region-specific and frequency-dependent manner

Local field potentials reflect cortical population dynamics in a region-specific and frequency-dependent manner

Recommendations for Responsible Development and Application of Neurotechnologies

Cortical control of virtual self-motion using task-specific subspaces

Contact Info

Product

Resources

About