A real-time, high-performance brain-computer interface for finger decoding and quadcopter control

Abstract

People with paralysis express unmet needs for peer support, leisure activities, and sporting activities. Many within the general population rely on social media and massively multiplayer video games to address these needs. We developed a high-performance finger brain-computer-interface system allowing continuous control of 3 independent finger groups with 2D thumb movements. The system was tested in a human research participant over sequential trials requiring fingers to reach and hold on targets, with an average acquisition rate of 76 targets/minute and completion time of 1.58 ± 0.06 seconds. Performance compared favorably to previous animal studies, despite a 2-fold increase in the decoded degrees-of-freedom (DOF). Finger positions were then used for 4-DOF velocity control of a virtual quadcopter, demonstrating functionality over both fixed and random obstacle courses. This approach shows promise for controlling multiple-DOF end-effectors, such as robotic fingers or digital interfaces for work, entertainment, and socialization.

A flexible intracortical brain-computer interface for typing using finger movements

“…noise). Mathematically, for the target velocity unit vector t̂ i , predicted velocity p i at the time step i , dSNR is given by (see (Willsey et al 2024) for further details): …”

“…Specifically, if successive characters in a sentence are on different finger groups, they can be simultaneously pointed to by corresponding fingers and then selected with a single click. This approach has recently shown high throughput in a center-out target acquisition task using virtual fingers (Willsey et al 2024). For the right-hand keyboard with three finger groups, 63% of trials had two successive characters on different finger groups for typing English sentences.…”

“…For example, while trained touch-typists can achieve speeds of up to 200 words per minute, novice typists can still use the full capability of a QWERTY keyboard with one finger. Motivated by the recent demonstrations of decoding dexterous finger movements from intracortical recordings in monkeys (Nason et al 2021; Willsey et al 2022; Costello et al 2023) and human participants (Wodlinger et al 2015; Jorge et al 2020; Vargas-Irwin et al 2022; Guan et al 2023; Shah et al 2023; Willsey et al 2024), we designed an intracortical BCI keyboard for typing based on flexion/extension movements of fingers. Focusing on finger movements simplifies the decoder algorithm compared with a QWERTY keyboard design, which would require decoding wrist movements in addition to finger movements.…”

“…Decoders were either pre-trained on other sessions and fine-tuned on the target session (purple and blue) or fine-tuned on the target session from scratch (red, random initialization of decoder parameters). Performance was evaluated using the directional signal-to-noise ratio (dSNR, (Willsey et al 2024)) of predictions on testing data. The solid line indicates the average dSNR across ten blocks and transparent lines indicate the standard error across blocks.…”

“…Second, as the finger movements are represented non-linearly in the motor cortical neural activity (Willsey et al 2022; Shah et al 2023), we used a multi-layered feedforward neural network to decode instantaneous finger velocities from the recorded neural activity (architecture shown in Fig. 2B, adapted from (Willsey et al 2024)). While real-time continuous control was also achievable by directly decoding finger positions, velocity decoding was faster (Fig.…”