High-performance brain-to-text communication via handwriting

Abstract

Brain-computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. To date, a major focus of BCI research has been on restoring gross motor skills, such as reaching and grasping 1-5 or point-and-click typing with a 2D computer cursor 6,7 . However, rapid sequences of highly dexterous behaviors, such as handwriting or touch typing, might enable faster communication rates. Here, we demonstrate an intracortical BCI that decodes attempted handwriting movements from neural activity in motor cortex and translates it to text in real-time, using a novel recurrent neural network decoding approach. With this BCI, our study

A Nonlinear Maximum Correntropy Information Filter for High-Dimensional Neural Decoding

“…In our study, the NN used as the nonlinear approximator is not limited to MLP but can also be RNN etc., as RNN is usually used in speech recognition and natural language processing. In the area of motor intention estimation, it has been recently used to classify the letter shapes using invasive brain signals [37]. However, the training of MLP is much simpler than the backpropagation through time in RNN.…”

“…However, a simple multi-layer perception (MLP) network could ignore the time dynamics in brain states. The recurrent neural network (RNN) model considers the possible internal dynamics and has been used to decode the letter shapes for classification [37]. The long short-term memory (LSTM) network has been used in the classification of the motor imagery task based on electroencephalography (EEG) signals [38].…”

Advancing sensory neuroprosthetics using artificial brain networks

“… 5 In another study, an implantable cortical interface was employed for high-performance brain-to-text communication via imagined handwriting long after motor function was lost. 6 Besides restoration of motor and cognitive functions, neuroprosthetics have also successfully restored sensory function, e.g., in the auditory domain using cochlear implants. 7 Similarly, simple vision could be restored using retinal implants, 8 but restoration of more detailed visual perception was challenging due to the intricate spatiotemporal patterns of retinal or optic nerve activity that encode such perception.…”

X‐ray scattering reveals disordered linkers and dynamic interfaces in complexes and mechanisms for DNA double‐strand break repair impacting cell and cancer biology

“…Yet, systematic analyses of their accuracy show these detailed structures can be too rigid versus functional solution structures. 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 Therefore for DNA repair and damage responses ranging from oxidized base repair to DNA double‐strand break (DSB) repair (DSBR), we have found that accurate measures of flexibility, conformational change, and dynamic complexes from small‐angle X‐ray scattering (SAXS) are often important for understanding and dissecting multifunctional mechanisms, as exemplified by the intrinsically disordered tail of Nei Like DNA Glycosylase 1 (NEIL1) acting in efficient oxidized base repair 13 , 14 and by ATP‐driven RAD50 assembly and conformational states acting in the homology‐directed repair (HDR) of DSBR. 15 , 16 , 17 , 18 , 19 , 20 …”

“…In double‐strand break repair (DSBR), there is a temporally coordinated assembly of proteins at DNA ends. 10 , 18 , 43 , 44 , 45 The DNA ends to be rejoined will need to be protected, held to keep a DSB from becoming a chromosome break, processed to make both ends suitable for ligation, and aligned for joining: this requires flexibility and dynamic assemblies in DSBR proteins and especially in their key scaffold proteins such as XRCC1 that enables alternative end‐joining for DSBR and replication restart. 1 , 46 Yet even in dynamic nucleotide excision repair (NER) assemblies, the extreme precision of the excised oligonucleotide supports TFIIH‐based licensing and ruler features that strictly dictate when and where the incision sites occur relative to the lesion.…”