A General LSTM-based Deep Learning Method for Estimating Neuronal Models and Inferring Neural Circuitry

Abstract

Computational neural models are essential tools for neuroscientists to study the functional roles of single neurons or neural circuits. With the recent advances in experimental techniques, there is a growing demand to build up neural models at single neuron or large-scale circuit levels. A long-standing challenge to build up such models lies in tuning the free parameters of the models to closely reproduce experimental recordings. There are many advanced machine-learning-based methods developed recently for parameter tuning, but many of them are task-specific or requires onerous manual interference. There lacks a general and fully-automated method since now. Here, we present a Long Short-Term Memory (LSTM)-based deep learning method, General Neural Estimator (GNE), to fully automate the parameter tuning procedure, which can be directly applied to both single neuronal models and large-scale neural circuits. We made comprehensive comparisons with many advanced methods, and GNE showed outstanding performance on both synthesized data and experimental data. Finally, we proposed a roadmap centered on GNE to help guide neuroscientists to computationally reconstruct single neurons and neural circuits, which might inspire future brain reconstruction techniques and corresponding experimental design. The code of our work will be publicly available upon acceptance of this paper.

Distilling multi-scale neural mechanisms from diverse unlabeled experimental data using deep domain-adaptive inference framework

“…b, Architecture of temporal feature extractor. It consists of a ResNet and an LSTM layer (Sheng et al, 2021). c, Feature visualization of the experimental datum in b .…”

“…This kind of inference is sometimes termed as simulation-based inference. Typical methods developed for the simulation-based inference range from evolutionary search (ES) algorithms (Druckmann et al, 2007; Gouwens et al, 2018; Gurkiewicz & Korngreen, 2007; Prinz et al, 2003), probabilistic modelling (Gonçalves et al, 2020; Oesterle et al, 2020; Schröder et al, 2019) to deep learning approaches (Ben-Shalom et al, 2019; Sheng et al, 2021). Most of the methods apply iterative simulations to find the model parameters that can produce the most similar model responses to experimental data.…”

“…Inferring neural properties from experimental data is crucial for revealing the underlying functional mechanisms (3, 51–54). Many ML methods have been proposed to tackle this problem by training a ML model on labeled synthetic data randomly generated from computational models and then directly apply the ML model to experimental data (14, 16, 31). In this study, we pointed out that these methods will encounter the OOD problem and hence produce inaccurate neural properties which can further result in a biased understanding of the mechanisms.…”

“…Finally, we implement our framework based on the ResNet-LSTM-FC architecture because of its outstanding performance on synthetic data (Sheng et al, 2021). In our previous work, we demonstrated that this architecture significantly outperforms the normal convolution neural network (CNN) architecture in many neural inference tasks on synthetic datasets (Sheng et al, 2021). Specifically, this DNN architecture has two key components: temporal feature extractor 𝑓 (ResNet-LSTM) and regressor or classifier 𝑔 (FC) (Fig.…”

“…To tackle this challenge, many advanced machine learning techniques have been introduced and made significant progress (Ben-Shalom et al, 2019; Endo et al, 2021; Gonçalves et al,2020; Sheng et al, 2021), as a result of their extraordinary abilities in dealing with complex high-dimensional data. The common approach used by them is synthesizing a training dataset with “ground-truth” labels generated by computational models and then applying the trained machine learning model directly to experimental data.…”