Teravoxel volume electron microscopy data sets from neural tissue can now be acquired in weeks, but data analysis requires years of manual labor. We developed the SyConn framework, which uses deep convolutional neural networks and random forest classifiers to infer a richly annotated synaptic connectivity matrix from manual neurite skeleton reconstructions by automatically identifying mitochondria, synapses and their types, axons, dendrites, spines, myelin, somata and cell types. We tested our approach on serial block-face electron microscopy data sets from zebrafish, mouse and zebra finch, and computed the synaptic wiring of songbird basal ganglia. We found that, for example, basal-ganglia cell types with high firing rates in vivo had higher densities of mitochondria and vesicles and that synapse sizes and quantities scaled systematically, depending on the innervated postsynaptic cell types.

Reconstruction and annotation of volume electron microscopy data sets of brain tissue is challenging but can reveal invaluable information about neuronal circuits. Significant progress has recently been made in automated neuron reconstruction as well as automated detection of synapses. However, methods for automating the morphological analysis of nanometer-resolution reconstructions are less established, despite the diversity of possible applications. Here, we introduce cellular morphology neural networks (CMNs), based on multi-view projections sampled from automatically reconstructed cellular fragments of arbitrary size and shape. Using unsupervised training, we infer morphology embeddings (Neuron2vec) of neuron reconstructions and train CMNs to identify glia cells in a supervised classification paradigm, which are then used to resolve neuron reconstruction errors. Finally, we demonstrate that CMNs can be used to identify subcellular compartments and the cell types of neuron reconstructions.

Learning turns experience into better decisions. A key problem in learning is credit assignment-knowing how to change parameters, such as synaptic weights deep within a neural network, in order to improve behavioral performance. Artificial intelligence owes its recent bloom largely to the error-backpropagation algorithm 1 , which estimates the contribution of every synapse to output errors and allows rapid weight adjustment. Biological systems, however, lack an obvious mechanism to backpropagate errors. Here we show, by combining high-throughput volume electron microscopy 2 and automated connectomic analysis [3][4][5] , that the synaptic architecture of songbird basal ganglia supports local credit assignment using a variant of the node perturbation algorithm proposed in a model of songbird reinforcement learning 6,7 . We find that key predictions of the model hold true: first, cortical axons that encode exploratory motor variability terminate predominantly on dendritic shafts of striatal spiny neurons, while cortical axons that encode song timing terminate almost exclusively on spines. Second, synapse pairs that share a presynaptic cortical timing axon and a postsynaptic spiny dendrite are substantially more similar in size than expected, indicating Hebbian plasticity 8,9 . Combined with numerical simulations, these findings provide strong evidence for a biologically plausible credit assignment mechanism 6 . Neural circuits that control decisions and actions are recurrently connected and involve many network layers from sensory inputs to motor output. Yet, as we learn, some mechanism specifies precisely which synapses, out of trillions, are to be modified and in what way. The backpropagation algorithm 10 is powerful because it directly calculates, based on the network architecture, the derivative of output errors with respect to every synaptic weight, providing an efficient method to update synaptic strengths. However, it remains unclear whether backpropagation or its variants are biologically implemented, or even plausible [11][12][13] . An alternative approach to implement gradient-based learning is weight-or node-perturbation 14,15 , in which the activity of a specific synapse or neuron is stochastically varied to determine its contribution to the output. Here we use a connectomic approach to study the biological implementation of stochastic gradient descent, which requires as-yet unknown circuit structures to inject variability, correlate variability with reward signals, and correctly assign credit to relevant synapses.Node perturbation is conceptually similar to behavioral trial-and-error reinforcement learning (RL) 16,17 . In the vertebrate brain RL is thought to involve the basal ganglia 18 , where a multitude of sensory and other context and state signals converge with action and outcome signals to determine which actions in which state lead to the best outcomes. In the songbird, the basal ganglia circuit dedicated to song learning 19 , Area X, receives synaptic input from two cortical areas 20 : the voc...

Dense reconstruction of synaptic connectivity requires high-resolution electron microscopy images of entire brains and tools to efficiently trace neuronal wires across the volume. To generate such a resource, we sectioned and imaged a larval zebrafish brain by serial block-face electron microscopy at a voxel size of 14 × 14 × 25 nm3. We segmented the resulting dataset with the flood-filling network algorithm, automated the detection of chemical synapses and validated the results by comparisons to transmission electron microscopic images and light-microscopic reconstructions. Neurons and their connections are stored in the form of a queryable and expandable digital address book. We reconstructed a network of 208 neurons involved in visual motion processing, most of them located in the pretectum, which had been functionally characterized in the same specimen by two-photon calcium imaging. Moreover, we mapped all 407 presynaptic and postsynaptic partners of two superficial interneurons in the tectum. The resource developed here serves as a foundation for synaptic-resolution circuit analyses in the zebrafish nervous system.

The ability to acquire ever larger datasets of brain tissue using volume electron microscopy leads to an increasing demand for the automated extraction of connectomic information. We introduce SyConn2, an open-source connectome analysis toolkit, which works with both on-site high-performance compute environments and rentable cloud computing clusters. SyConn2 was tested on connectomic datasets with more than 10 million synapses, provides a web-based visualization interface and makes these data amenable to complex anatomical and neuronal connectivity queries.

Reconstruction and annotation of volume electron microscopy data sets of brain tissue is challenging, but can reveal invaluable information about neuronal circuits. Significant progress has recently been made in automated neuron reconstruction, as well as automated detection of synapses. However, methods for automating the morphological analysis of nanometer-resolution reconstructions are less established, despite their diverse application possibilities. Here, we introduce cellular morphology neural networks (CMNs), based on multi-view projections sampled from automatically reconstructed cellular fragments of arbitrary size and shape. Using unsupervised training we inferred morphology embeddings ("Neuron2vec") of neuron reconstructions and trained CMNs to identify glia cells in a supervised classification paradigm which was used to resolve neuron reconstruction errors. Finally, we demonstrate that CMNs can be used to identify subcellular compartments and the cell types of neuron reconstructions.