Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity

Pei, Felix; Ye, Joel; Zoltowski, David L.; Wu, Anqi; Chowdhury, Raeed H.; Sohn, Hansem; O’Doherty, Joseph E.; Shenoy, Krishna V.; Kaufman, Matthew T.; Churchland, Mark M.; Jazayeri, Mehrdad; Miller, Lee E.; Park, Il; Dyer, Eva L.; Pandarinath, Chethan

doi:10.48550/arxiv.2109.04463

Cited by 14 publications

(24 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This dataset contains 108 different reach configurations over nearly 3000 trials, and has recently been proposed as a neuroscience benchmark for neural data analysis methods (Pei et al, 2021). We compared the performance of iLQR-VAE to several other latent variable models, evaluated on this dataset in Pei et al (2021).…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Conditioning of 137 neurons (i.e using 45 held-out neurons), we obtained a co-smoothing of 0.331 ± 0.001 (over 5 random seeds). For comparison, Pei et al (2021) reports 0.187 for GPFA (Yu et al, 2009), 0.225 for SLDS (Linderman et al, 2017), 0.329 for Neural Data Transformers (Ye and Pandarinath, 2021) and R 2 = 0.346 for AutoLFADS (LFADS with large scale hyperparameter optimization; Keshtkaran et al, 2021) on the same dataset.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…To highlight the utility of iLQR-VAE for analyzing experimental neuroscience data, we next applied it to recordings of monkey motor (M1) and dorsal premotor (PMd) cortices during a delayed reaching task (‘Maze’ dataset of Kaufman et al, 2016; DANDI 000128). This dataset contains 108 different reach configurations over nearly 3000 trials, and has recently been proposed as a neuroscience benchmark for neural data analysis methods (Pei et al, 2021). We compared the performance of iLQR-VAE to several other latent variable models, evaluated on this dataset in Pei et al (2021).…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…As the test data used in the NLB challenge is not publicly available, the results we reported were not computed on the exact same data split. However, the model performed highly consistently across random seeds, such that we expect iLQR-VAE’s performance to be directly comparable to the results from Pei et al (2021). To fit these data, we ran iLQR-VAE on 168 CPUs for ∼ 6h, using a mini-batch size of 168 trials.…”

Section: Additional Related Workmentioning

confidence: 96%

“…To allow for direct comparison with benchmarks reported Pei et al (2021), we first used data provided by the Neural Latents Benchmark (NLB) challenge, available at https://gui.dandiarchive.org/#/dandiset/000128.…”

Section: Additional Related Workmentioning

confidence: 99%

See 4 more Smart Citations

iLQR-VAE : control-based learning of input-driven dynamics with applications to neural data

Schimel

Kao

Jensen

et al. 2021

Preprint

View full text Add to dashboard Cite

Understanding how neural dynamics give rise to behaviour is one of the most fundamental questions in systems neuroscience. To achieve this, a common approach is to record neural populations in behaving animals, and model these data as emanating from a latent dynamical system whose state trajectories can then be related back to behavioural observations via some form of decoding. As recordings are typically performed in localized circuits that form only a part of the wider implicated network, it is important to simultaneously learn the local dynamics and infer any unobserved external input that might drive them. Here, we introduce iLQR-VAE, a novel control-based approach to variational inference in nonlinear dynamical systems, capable of learning both latent dynamics, initial conditions, and ongoing external inputs. As in recent deep learning approaches, our method is based on an input-driven sequential variational autoencoder (VAE). The main novelty lies in the use of the powerful iterative linear quadratic regulator algorithm (iLQR) in the recognition model. Optimization of the standard evidence lower-bound requires differentiating through iLQR solutions, which is made possible by recent advances in differentiable control. Importantly, having the recognition model implicitly defined by the generative model greatly reduces the number of free parameters and allows for flexible, high-quality inference. This makes it possible for instance to evaluate the model on a single long trial after training on smaller chunks. We demonstrate the effectiveness of iLQR-VAE on a range of synthetic systems, with autonomous as well as input-driven dynamics. We further show state-of-the-art performance on neural and behavioural recordings in non-human primates during two different reaching tasks.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Additional Related Workmentioning

confidence: 96%

Section: Additional Related Workmentioning

confidence: 99%

See 3 more Smart Citations

iLQR-VAE : control-based learning of input-driven dynamics with applications to neural data

Schimel

Kao

Jensen

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Disentangling the flow of signals between populations of neurons

et al. 2022

View full text Add to dashboard Cite

Technological advances now allow us to record from large populations of neurons across multiple brain areas. These recordings may illuminate how communication between areas contributes to brain function, yet a substantial barrier remains: How do we disentangle the concurrent, bidirectional flow of signals between populations of neurons? We therefore propose here a novel dimensionality reduction framework: Delayed Latents Across Groups (DLAG). DLAG disentangles signals relayed in each direction, identifies how these signals are represented by each population, and characterizes how they evolve within and across trials. We demonstrate that DLAG performs well on synthetic datasets similar in scale to current neurophysiological recordings. Then

show abstract

Benchmarking of hardware-efficient real-time neural decoding in brain–computer interfaces

Hueber,

Tang,

Sifalakis

et al. 2024

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

Designing processors for implantable closed-loop neuromodulation systems presents a formidable challenge due to the constrained operational environment, requiring low latency and high energy efficiency. Previous benchmarks have provided limited insights into energy efficiency and latency. This paper, however, introduces algorithmic metrics that capture the potential and limitations of neural decoders for closed-loop intra-cortical brain-computer interfaces in the context of energy and hardware constraints. This study benchmarks common decoding methods for predicting a primate’s finger kinematics from the motor cortex, and explores the suitability for low latency and low compute neural decoding. The study finds that ANN-based decoders provide superior decoding accuracy, requiring a high latency and many operations to decode neural signals effectively. Spiking neural networks emerge as a solution, bridging this gap by achieving competitive decoding performance within sub-10ms while utilizing a fraction of the computational resources. This distinctive advantages of neuromorphic spiking neural networks, positions them as highly suitable for the challenging environment of closed-loop neural modulation. Their capacity to balance decoding accuracy and operational efficiency offers immense potential in reshaping the landscape of neural decoders, fostering greater understanding, and opening new frontiers in closed-loop intracortical human-machine interaction.

show abstract

Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity

Cited by 14 publications

References 57 publications

iLQR-VAE : control-based learning of input-driven dynamics with applications to neural data

iLQR-VAE : control-based learning of input-driven dynamics with applications to neural data

Disentangling the flow of signals between populations of neurons

Benchmarking of hardware-efficient real-time neural decoding in brain–computer interfaces

Contact Info

Product

Resources

About