An emerging view of neural geometry in motor cortex supports high-performance decoding

Abstract

Decoders for brain-computer interfaces (BCIs) assume constraints on neural activity, chosen to reflect scientific beliefs while yielding tractable computations. We document how low tangling – a typical property of motor-cortex neural trajectories – yields unusual neural geometries. We designed a decoder, MINT, to embrace appropriate statistical constraints for these geometries. MINT takes a trajectory-centric approach: a library of neural trajectories (rather than a set of neural dimensions) provides a scaffold approximating the neural manifold. Each neural trajectory has a corresponding behavioral trajectory, allowing straightforward but highly nonlinear decoding. MINT outperformed other interpretable methods, and outperformed expressive machine learning methods in 37 of 42 comparisons. Yet unlike such methods MINT's constraints are known, not the implicit result of optimizing decoder output. MINT performed well across tasks, suggesting its assumptions are generally well-matched to the statistics of neural data. Despite embracing highly nonlinear relationships between behavior and potentially complex neural trajectories, MINT's computations are simple, scalable, and provide interpretable quantities such as data likelihoods. MINT's performance and simplicity suggest it may be an excellent candidate for clinical BCI applications.

Low-dimensional neural manifolds for the control of constrained and unconstrained movements

“…However, the population neural dynamics mostly explored nearly linear regions of the neural manifolds. This finding is consistent with our results demonstrating the ability to linearly decode EMG signals from relatively few linear latent variables, and provides a manifoldbased understanding of the surprising effectiveness of linear methods in decoding the motor output in brain-machine interfaces 42,43,60,80 .…”

“…Linear decoding is based on a weighted sum of neural signals; it is thus the projection of collective neural activity along a specific direction in the neural state space. Although this widely used approach 42,43,61 has been superseded by nonlinear and recurrent alternatives 43,[79][80][81] , linear decoders are simple, interpretable, and effective, and provide a useful tool for hypothesis testing. In this study, we were interested not in the absolute EMG decoding accuracy achieved when all neurons are used as inputs, but in the relative accuracy achieved when only the latent variables associated with the low-dimensional manifolds are used as inputs.…”

Thoughtful faces: inferring internal states across species using facial features

“…The MSLR manages to reflect the non-stationarity and regime shifts often present in behavioural data (43– 46), by flexibly accommodating complex temporal dynamics while keeping a relative simplicity, compared to deep learning-based methods (47). Moreover, the MSLR is less data-hungry than other common data-driven models (48–50). The MSLR uses the ‘pre-stimulus’ facial features to predict the animals’ reaction time (RT) by assuming ‘hidden’ states.…”

Bayesian inference of structured latent spaces from neural population activity with the orthogonal stochastic linear mixing model

“…Similar to LFADS, NDT refers to Neural Data Transformers, a non-recurrent neural network similar to LFADS that replaces the RNN encoder decoder with Transformer instead [52]. MINT refers to "mesh of idealized neural trajectories" [53]. It simply uses a state-transition lookup table to model the latent dynamics and proposes two conditional functions to jointly model both spiking activity and the behavioral statistics.…”