Artificial neural networks for neuroscientists: A primer

Abstract

Artificial neural networks (ANNs) are essential tools in machine learning that are increasingly used for building computational models in neuroscience. Besides being powerful techniques for data analysis, ANNs provide a new approach for neuroscientists to build models that capture complex behaviors, neural activity and connectivity, as well as to explore optimization in neural systems. In this pedagogical Primer, we introduce conventional ANNs and demonstrate how they have been deployed to study neuroscience questions. Next, we detail how to customize the analysis, structure, and learning of ANNs to better address a wide range of challenges in brain research. To help the readers garner hands-on experience, this Primer is accompanied with tutorial-style code in PyTorch and Jupyter Notebook, covering major topics. Artificial neural networks in neuroscienceArtificial neural networks (ANNs), or deep neural networks, have emerged as a dominant framework in machine learning (ML) nowadays [LeCun et al., 2015], leading to breakthroughs across a wide range of applications, including computer vision [Krizhevsky et al., 2012], natural language processing [Devlin et al., 2018], and strategic games [Silver et al., 2017]. Some key ideas in this field can be traced to brain research: supervised learning rules have its root in the theory of training perceptrons which in turn was inspired by the brain [Rosenblatt, 1962]; the hierarchical architecture [Fukushima and Miyake, 1982] and convolutional principle [LeCun and Bengio, 1995] were closely linked to our knowledge about the primate visual system Wiesel, 1962, Felleman andVan Essen, 1991]. Today, there is a continued exchange of ideas from neuroscience to the field artificial intelligence [Hassabis et al., 2017].At the same time, machine learning offers new and powerful tools for systems neuroscience. One utility of the deep learning framework is to analyze neuroscientific data (Figure 1A). In particular, the advances in computer vision makes convolutional neural networks useful for processing a range of image and video data. Behaviors over time, for instance micro-movements of animals in a laboratory experiment, are Preprint.

Slow manifolds in recurrent networks encode working memory efficiently and robustly

“…1a,b). All other temporal epochs are unconstrained, thus obviating the need to generate an error signal continuously throughout trials, which may overly constrain the dynamics 34 (see also Methods for additional details and Supplementary Fig. 1).…”

“…Further, in the IFC mechanism, most neurons are lightly saturated (Fig. 4c), meaning that most neurons are within a linear regime, as thought to occur in actual neural circuits 34,38 .…”

Intelligent Robotics in Pediatric Cooperative Neurorehabilitation: A Review

“…LSTMs have proved to be very powerful models and are at the core of most state-of-the-art temporal models (e.g., sentiment analysis in artificial emotional intelligence [24]). Well-rounded guides to artificial neural networks are available [25,26]. More recently, attention-based RNNs were shown to be exceptionally relevant for applications ranging from handwriting synthesis to speech recognition [27].…”