Deep Multi-State Dynamic Recurrent Neural Networks Operating on Wavelet Based Neural Features for Robust Brain Machine Interfaces

Abstract

We present a new deep multi-state Dynamic Recurrent Neural Network (DRNN) architecture for Brain Machine Interface (BMI) applications. Our DRNN is used to predict Cartesian representation of a computer cursor movement kinematics from open-loop neural data recorded from the posterior parietal cortex (PPC) of a human subject in a BMI system. We design the algorithm to achieve a reasonable trade-off between performance and robustness, and we constrain memory usage in favor of future hardware implementation. We feed the predictions of the network back to the input to improve prediction performance and robustness. We apply a scheduled sampling approach to the model in order to solve a statistical distribution mismatch between the ground truth and predictions. Additionally, we configure a small DRNN to operate with a short history of input, reducing the required buffering of input data and number of memory accesses. This configuration lowers the expected power consumption in a neural network accelerator. Operating on wavelet-based neural features, we show that the average performance of DRNN surpasses other state-of-the-art methods in the literature on both single-and multi-day data recorded over 43 days. Results show that multi-state DRNN has the potential to model the nonlinear relationships between the neural data and kinematics for robust BMIs.Recently, nonlinear machine learning algorithms have shown promise in attaining high performance and robustness in BMIs. For instance, Wessberg et al.[5] apply a fully-connected neural network to neural data recorded from a monkey. Shpigelman et al. [6] show that a Gaussian kernel outperforms a linear kernel in a Kernel Auto-Regressive Moving Average (KARMA) algorithm when decoding 3D kinematics from macaque neural activity. Sussillo et al. [7] apply a large FORCE Dynamic Recurrent Neural Network (F-DRNN) on neural data recorded from the primary motor cortex in two monkeys, and then they test the stability of the model over multiple days [8]. Zhang et al. [9] and Schwemmer et al. [10] extract wavelet based features of motor cortex neural data of a human subject to classify intended hand movements by using a nonlinear support vector machine (SVM) and a large deep neural network, respectively. Hosman et al. [11] pass motor cortex neural firing rates to an LSTM and a Kalman filter to compare their performances for decoding intended cursor velocity of a human subject. These nonlinear learning-based decoders have shown more stability over multiple days and have improved performance compared to prior linear methods. However, they all have been applied to motor cortex data by mostly using neural firing rates as input features, which show more variability over long periods [2]. Recent work has demonstrated that neural activity in the posterior parietal cortex (PPC) can be used to support BMIs [12,13,14,15,16,17,18], although the encoding of movement kinematics appears to be complex. PPC processes a rich set of high-level aspects of movement including sensory integration, planning, an...

Real-time brain-machine interface in non-human primates achieves high-velocity prosthetic finger movements using a shallow feedforward neural network decoder

“…In open-loop testing, neural activity is obtained when the able-bodied animal performs a finger task with the hand manipulandum. From this pre-recorded neural activity, the position/ velocity of the fingers is predicted and compared with the true finger position/velocity from the pre-recorded task, and many sophisticated non-linear and neural network architectures are known to show tremendous promise and even outperform linear algorithms in open-loop mode 16,17,31,32 . In closed-loop mode, the decoding algorithm interprets neural activity in real-time and updates the position and velocity of the prosthesis.…”

Translating deep learning to neuroprosthetic control

“…Deep learning algorithms are being increasingly used to improve the performance of real-time BCIs 28,32,36,37,59,60 . Prior work investigating deep learning methods for BCIs has reported promising offline results 33,[54][55][56][61][62][63][64] , although most remain to be evaluated in an online setting due, in part, to the rarity of human BCI data. Here, we tested a deep learning method for real-time BCI control by a person with paralysis.…”

Data-efficient Causal Decoding of Spiking Neural Activity using Weighted Voting

“…The modular nature of our pipeline enables us to inherent the advantages of the classifier of choice. In particular, we used two popular classifiers (KNN and Poisson) that are very dataefficient compared to some other alternatives such as data-heavy deep learning models [20]. That is why, for each session, we were able to provide high accuracy with around only 140 seconds of training recordings (80% of the available 175 seconds of data from fifty trials each lasting 3500 ms).…”

“…Also, due to the highly dynamic nature of neural activities, parameters of the estimation model that is fitted to the data will need to be continually re-tuned. Although deep-learning methods that use a large number of model parameters have shown promising results in decoding offline datasets [18], [19], currently, these methods are less attractive for real-time applications with a large repertoire of tasks. The data-expensive and computationally heavy learning phase make them hard to be trained and run in real-time.…”