Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning

Abstract

Robustness and decoding accuracy remain major challenges in the clinical translation of intracortical brain-machine interface (BMI) systems. In this work, we show that a signal/decoder co-design methodology (exploiting the synergism between the input signal and decoding algorithm within the design development process) can be used to yield robust and accurate BMI decoding performance. Specifically, through applying this process, we propose the combination of using entire spiking activity (ESA) as the input signal and quasi-recurrent neural network (QRNN) based deep learning as the decoding algorithm. We evaluated the performance of ESA-driven QRNN decoder for decoding hand kinematics from neural signals chronically recorded from the primary motor cortex area of a non-human primate. Our proposed method yielded consistently higher decoding performance than any other methods previously reported across long-term recording sessions. Its high decoding performance could sustain, even when spikes were removed from the raw signals.Overall results demonstrate exceptionally high decoding accuracy and chronic robustness, which is highly desirable given it is an unresolved challenge in BMIs. this issue is by utilising a different type of input signals, namely multiunit activity (MUA). MUA, defined as all spikes detected through threshold crossing without spike sorting, offers simpler processing while providing better signal stability over time than SUA. 25,29,30 Another alternative input signal is local field potential (LFP), which is thought to mainly reflect summed synaptic activity from a local neuronal population around the recording electrodes. It is believed and has been demonstrated by experimental studies that LFPs exhibit better long-term signal stability than their spike counterparts. [31][32][33][34] Moreover, LFPs can be obtained by simpler processing and lower sampling rate that can reduce the power consumption of BMIs. Despite the appealing advantages of MUA and LFP, a considerable number of published studies have reported that the decoding accuracy of MUA 24,29,30,35, 36 and LFP 33,[36][37][38][39] are lower than that of SUA. Therefore, it is highly desirable to have an input signal that is not only stable but also yields high decoding accuracy.The decoding accuracy is also affected by the decoder, that is, an algorithm used to convert the input signal from the brain into the behavioural parameter of interest (e.g. hand kinematics). Many BMI studies employ linear decoders, e.g. Wiener filter (WF) 5,40,41 and Kalman filter (KF), 6,9,21,32,33,42 which could yield suboptimal decoding accuracy as neural signals are known to exhibit nonlinear and nonstationary properties. 43 Although there exists a nonlinear extension of WF and KF, called Wiener cascade filter (WCF) 19,39,44 and unscented Kalman filter (UKF), 45,46 WCF and UKF assume that the noise is (additive) stationary and Gaussian, respectively. If these a priori assumptions are violated, the decoding performance of both decoders will not be optimal.The rise of ...

Inferring entire spiking activity from local field potentials

“…in fewer number of LFP channels) than SUA and MUA. This may suggest that ESA exhibits larger spatial coverage (encompassing smaller and farther neurons), which in turn contains more spiking information and higher interchannel correlation 23 . We also found that the higher LFP interchannel correlation (information redundancy), the quicker the inference performance of spiking activity reaches its plateau.…”

“…receptive field) in the visual cortex of monkeys while receiving various visual stimuli 14 , 15 , 20 – 22 . ESA has also been shown to yield more accurate and robust decoding of hand kinematics compared to SUA and MUA from three monkeys performing different tasks 23 .…”

The Significance of Neural Inter-Frequency Correlations

“…To give a very few, non-exhaustive examples: [1,2] investigated the ability of different Local Field Potential (LFP) power bands to decode various movements. In [3], they decoded a hand-reaching task using different time-frequency features of intracortical neural signals, e.g. LFP, Single-Unit Activity (SUA), MUA, Entire Spiking Activity (ESA), using linear and deep learning methods.…”

Deep learning approaches for neural decoding across architectures and recording modalities

“…For example, with binary spiking events as input, the best size and temporal placement of time bins could be automatically determined, or even features related to the precise timing of spikes could be learned. It was also recently shown that using the envelope of spiking activity, a continuous signal, followed by feature extraction within a neural network, was able to improve decoding performance [48].…”