Petascale neural circuit reconstruction: automated methods

Abstract

3D electron microscopy (EM) has been successful at mapping invertebrate nervous systems, but the approach has been limited to small chunks of mammalian brains. To scale up to larger volumes, we have built a computational pipeline for processing petascale image datasets acquired by serial section EM, a popular form of 3D EM. The pipeline employs convolutional nets to compute the nonsmooth transformations required to align images of serial sections containing numerous cracks and folds, detect neuronal boundaries, label voxels as axon, dendrite, soma, and other semantic categories, and detect synapses and assign them to presynaptic and postsynaptic segments. The output of neuronal boundary detection is segmented by mean affinity agglomeration with semantic and size constraints. Pipeline operations are implemented by leveraging distributed and cloud computing. Intermediate results of the pipeline are held in cloud storage, and can be effortlessly viewed as images, which aids debugging. We applied the pipeline to create an automated reconstruction of an EM image volume spanning four visual cortical areas of a mouse brain. Code for the pipeline is publicly available, as is the reconstructed volume.

Perisomatic Features Enable Efficient and Dataset Wide Cell-Type Classifications Across Large-Scale Electron Microscopy Volumes

“…The different cell types of neocortex vary in their complex dendritic and axonal morphologies, [1][2][3][4][5][6][7] along with their physiological properties 8,9 and molecular expression profiles. [10][11][12][13] Large scale electron microscopy (EM) data with automated reconstructions [14][15][16][17] creates both opportunities and challenges for examining the structural variation of cell types across cortex. EM provides an opportunity to study a host of morphological and subcellular detail that has not historically been studied at a large quantitative scale.…”

Petascale pipeline for precise alignment of images from serial section electron microscopy

“…Even our preliminary pipeline was able to precisely correct such large discontinuous distortions, ensuring that neurites at the boundary of the defect were still well-aligned. This led to a successful automated reconstruction (Macrina et al 2021). The alignment took 230 hours on a cluster of preemptible NVIDIA T4 GPUs on Google Cloud.…”

“…Samples of manually annotated cracks and folds in cutouts of 1024 × 1024 px at 64 nm resolution, were collected from the original data and were used to supervise training of separate UNets as described in (Macrina et al, 2021). A tissue segmentation model was trained in similar fashion.…”

MorphoFeatures: unsupervised exploration of cell types, tissues and organs in volume electron microscopy

“…For the first time, the rapid progress in volume EM allows this for increasingly large samples, including organs and entire smaller animals, with nanometer resolution ( Bae et al (2021) ; Vergara et al (2021) ; Zheng et al (2018) ; Scheffer et al (2020) ; Cook et al (2019) ; Verasztó et al (2020) ). Supported by automated cell segmentation pipelines ( Heinrich et al (2021) ; Macrina et al (2021) ; Pape et al (2017) ; Müller et al (2021) ), such datasets allow characterization and comparison of cellular morphologies, including membrane-bound and membraneless organelles and inclusions, at an unprecedented level of detail for thousands of cells simultaneously ( Turner et al (2022) ; Vergara et al (2021) ). A notable challenge of such large-scale studies is the analysis of massive amounts of imaging data that can no longer be inspected manually.…”