Inferring single-trial neural population dynamics using sequential auto-encoders

Abstract

Neuroscience is experiencing a data revolution in which simultaneous recording of many hundreds or thousands of neurons is revealing structure in population activity that is not apparent from single-neuron responses. This structure is typically extracted from trial-averaged data. Single-trial analyses are challenging due to incomplete sampling of the neural population, trial-to-trial variability, and fluctuations in action potential timing. Here we introduce Latent Factor Analysis via Dynamical Systems (LFADS), a deep learning method to infer latent dynamics from single-trial neural spiking data. LFADS uses a nonlinear dynamical system (a recurrent neural network) to infer the dynamics underlying observed population activity and to extract 'de-noised' single-trial firing rates from neural spiking data. We apply LFADS to a variety of monkey and human motor cortical datasets, demonstrating its ability to predict observed behavioral variables with unprecedented accuracy, extract precise estimates of neural dynamics on single trials, infer perturbations to those dynamics that correlate with behavioral choices, and combine data from non-overlapping recording sessions (spanning months) to improve inference of underlying dynamics. In summary, LFADS leverages all observations of a neural population's activity to accurately model its dynamics on single trials, opening the door to a detailed understanding of the role of dynamics in performing computation and ultimately driving behavior.Increasing evidence suggests that in many brain areas, such as the motor and prefrontal cortices, the activity of large populations of neurons, termed the neural population state, is often well-described by low-dimensional dynamics [e.g. (Afshar et al. 2011; Harvey, Coen, and Tank 2012; Kaufman et al. 2014;Sadtler et al. 2014;Kobak et al. 2016a) ]. Recovering these dynamics on single trials is essential for illuminating the relationship between neural population activity and behavior, and for advancing therapeutic neurotechnologies such as closed-loop deep brain stimulation and brain-machine interfaces. However, recovering population dynamics All rights reserved. No reuse allowed without permission.(which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint . http://dx.doi.org/10.1101/152884 doi: bioRxiv preprint first posted online Jun. 20, 2017; on single trials is difficult due to trial-to-trial variability (e.g. in behavior or arousal) and fluctuations in the spiking of individual neurons. Even with dramatic increases in the numbers of neurons that can be simultaneously recorded using multichannel electrode arrays or optical imaging, accurately recovering population dynamics from single trials remains a significant challenge for data-analysis methods.Standard analyses sacrifice single-trial information for the sake of better estimates of trial-averaged neural states (Ahrens et al. 2012; Kobak et al. 2016b) . Techniques for extrac...

Deep Multi-State Dynamic Recurrent Neural Networks Operating on Wavelet Based Neural Features for Robust Brain Machine Interfaces

“…We evaluate the performance of our DRNN on 12 neural features: High-frequency, Mid-frequency, and Low-frequency Wavelet features (HWT, MWT, LWT); High-frequency, Mid-frequency, and Low-frequency Fourier powers (HFT, MFT, LFT); Latent Factor Analysis via Dynamical Systems (LFADS) features [23]; High-Pass and Low-Pass Filtered (HPF, LPF) data; Threshold Crossings (TCs); Multi-Unit Activity (MUA); and combined MWT and TCs (MWT + TCs) ( Table 1).…”

“…LFADS is a generalization of variational auto-encoders that can be used to model time-varying aspect of neural signals. Pandarinath et al [23] shows that decoding performance improved when using LFADS to infer smoothed and denoised firing rates. We used LFADS to generate features based on the trial-by-trial threshold crossings from each center-out task.…”

Precise timing is ubiquitous, consistent and coordinated across a comprehensive, spike-resolved flight motor program

“…For example, mo-67 tor coordination patterns across muscles (e.g. muscle synergies 68 (31)) and population recordings of M1 neurons in motor cortex 69 (32) all consider how populations of units encode movement 70 through spike rate. Alternatively, spike timing codes may 71 play a role in the coordination of muscles in motor systems.…”

Novel neural signal features permit robust machine-learning of natural tactile- and proprioception-dominated dorsal column nuclei signals

“…We previously devised a metric for quantifying the relevance of neural signal features to the stimulus from which they were evoked, which we termed feature-learnability (Loutit et al, 2019). This approach uses a simple feedforward back-propagation supervised artificial neural network (ANN), which has several advantages over the use of state-of-the-art deep neural network (DNN) architectures (Pandarinath et al, 2018): An ANN enables the quantification of information content from signal features of interest, that, for example, may be selected on the basis of: i) their neurophysiological relevance, ii) their capacity to mimic or inform electrical stimulation in sensory applications, and/or iii) are commonly used for decoding neural signals. DNNs would not permit us to directly test specific features of interest, but rather, would identify abstract features that may have little identifiable relevance.…”