VESICLE: Volumetric Evaluation of Synaptic Inferfaces using Computer Vision at Large Scale

Abstract

An open challenge at the forefront of modern neuroscience is to obtain a comprehensive mapping of the neural pathways that underlie human brain function; an enhanced understanding of the wiring diagram of the brain promises to lead to new breakthroughs in diagnosing and treating neurological disorders. Inferring brain structure from image data, such as that obtained via electron microscopy (EM), entails solving the problem of identifying biological structures in large data volumes. Synapses, which are a key communication structure in the brain, are particularly difficult to detect due to their small size and limited contrast. Prior work in automated synapse detection has relied upon time-intensive, error-prone biological preparations (isotropic slicing, post-staining) in order to simplify the problem. This paper presents VESICLE, the first known approach designed for mammalian synapse detection in anisotropic, non-poststained data. Our methods explicitly leverage biological context, and the results exceed existing synapse detection methods in terms of accuracy and scalability. We provide two different approaches -a deep learning classifier (VESICLE-CNN) and a lightweight Random Forest approach (VESICLE-RF), to offer alternatives in the performance-scalability space. Addressing this synapse detection challenge enables the analysis of high-throughput imaging that is soon expected to produce petabytes of data, and provides tools for more rapid estimation of brain-graphs. Finally, to facilitate community efforts, we developed tools for large-scale object detection, and demonstrated this framework to find ⇡ 50,000 synapses in 60,000 µm 3 (220 GB on disk) of electron microscopy data.Figure 1: Previous work on synapse detection has focused on isotropic post-stained data (left), which shows crisp membranes and dark fuzzy post synaptic densities (arrows) from all orientations. The alternative imaging technique of non post-stained, anisotropic data (middle, right) promises higher throughput, lack of staining artifacts, reduction in lost slices, and less demanding data storage requirements -all critically important for high-throughput connectomics. The XZ plane of a synapse in anisotropic data is shown (right), illustrating the effect of lower resolution. We address this more challenging environment, in which membranes appear fuzzier and are harder to distinguish from synaptic contacts. Data courtesy of Graham Knott (left) and Jeff Lichtman (middle, right).In Figure 1, we provide examples demonstrating the challenging setting of our detection task. Figure 2 illustrates the importance of leveraging contextual information, especially neurotransmitter-containing vesicles and cell membranes. Finally, Figure 3 presents a summary of classifier performance; visualizations of the results are not shown here, but are available in the full paper and on our website. Our code and data are open source and available at: openconnecto.me/vesicle. The detection task is to identify synapses shown in green (upper right). These synapses are kn...

Detecting Synapse Location and Connectivity by Signed Proximity Estimation and Pruning with Deep Nets

“…Figure 3(a) compares the precision recall curves for detecting both location and connectivity with two variants. 1) 3 Label + pruning -where the proximity approximation is replaced by 3-class classification among pre-, post-synaptic, and rest of the voxels (Figure 1 This experiment suggests that the pruning network is substantially more effective than morphological post-processing [23]. The proposed signed proximity approximation yields 3% more true positives in the initial candidate set than those generated by the multiclass prediction.…”

“…These algorithms assumed subsequent human intervention to determine the synaptic partners given the cleft predictions. Roncal et al [23] combine the information provided by membrane and vesicle predictions with a CNN (not fully convolutional) and apply post-processing to locate synaptic clefts. To establish the pre-and post-synaptic partnership, [24] augmented the synaptic cleft detection with a multilayer perceptron operating on hand designed features.…”

Toward A Reproducible, Scalable Framework for Processing Large Neuroimaging Datasets

“…Examples of tools for membrane segmentation include CNN [30] and U-nets [29] approaches. Synapse detection has been achieved using deep learning techniques and random forest classifiers [31,32]. Neural segmentation has been previously done using agglomeration-based approaches [33] and automated selection of neural networks [9].…”

Flexible Learning-Free Segmentation and Reconstruction for Sparse Neuronal Circuit Tracing

“…Neural networks can take weeks or months to train, and feature extraction, while faster than training, is still inefficient even on a graphical processing unit (GPU). For just 100 images with a resolution of 1024 × 1024, a state-of-the-art network for connectomics takes 59 hours for training and evaluation 29 .…”

“…A wide variety of methods -highly dependent on supervised machine learning -exist for the dense EM segmentation and reconstruction problem. The methods include SVM-based algorithms [12][13][14][15][16] , Random Forests 12, 17-24 , Conditional Random Fields 22 , and Artificial Neural Networks [25][26][27][28][29][30] (i.e., deep learning). These machine learning approaches can be found in popular software packages for connectomics image analysis such as Rhoana 31 and Ilastik 18,19,32 .…”