A large-scale neural network training framework for generalized estimation of single-trial population dynamics

High-performance neural population dynamics modeling enabled by scalable computational infrastructure

“…In Figure 3, inferred firing rates from the KubeFlow trained AutoLFADS model are shown along with conventional firing rate estimation strategies. Qualitatively, these example inferences are similar to those described in Keshtkaran et al (2022), showing similar consistency across trials and resemblance to peristimulus time histograms (PSTH). In Figure 2, we plot the hyperparameter and associated loss values for the KubeFlow based implementation of AutoLFADS to provide a visualization of the PBT based optimization process on these data.…”

“…One model was trained with the Ray solution and the other with the KubeFlow solution using matching PBT hyperparameters and model configurations to ensure that models of comparable quality can be learned across both solutions. A comprehensive description of the AutoLFADS algorithm and results applying the algorithm to neural data using Ray can be found in Keshtkaran et al (2022). We demonstrate similar converged model performances on metrics relevant to the quality of inferred firing rates in Table 2 (Pei et al, 2021).…”

“…Latent factor analysis via dynamical systems (LFADS) (Pandarinath et al, 2018) provides a deep learning approach for extracting estimates of these latent dynamics from neural population data. The recently developed AutoLFADS framework (Keshtkaran et al, 2022) extends LFADS by using Population Based Training (PBT) (Jaderberg et al, 2017) to effectively and scalably tune model hyperparameters, a critical step for accurate modeling of neural population data. As hyperparameter sweeps are one of the most computationally demanding processes in model development, these workflows should be deployed in a computationally efficient and cost effective manner given the compute resources available (e.g., local, institutionally-supported, or commercial computing clusters).…”

Expressive architectures enhance interpretability of dynamics-based neural population models

“…We also introduce a metric, state R 2 , which measures the fraction of inferred latent state variance explained by an affine transformation of the true latent states. Despite their success in reconstructing neural activity patterns [17,18,13], we find that RNN-based SAEs require many more latent dimensions than the synthetic systems they are attempting to model. Moreover, we find that the dynamics learned by the RNNs are a poor match to the synthetic systems, in that a large fraction of the models' variance reflects activity not seen in the synthetic system.…”

“…With the resurgence of deep learning over the past decade, a powerful class of methods has emerged that use RNNs to approximate f [17,18,30,13]. In head-to-head comparisons, RNN-based methods replicate neural activity patterns with substantially higher accuracy than LDSs on datasets from a variety of brain areas and behaviors, suggesting that linear dynamics may not adequately model the dynamics of neural systems [23].…”