Multi-Layered Maps of Neuropil with Segmentation-Guided Contrastive Learning

Abstract

Maps of the nervous system that identify individual cells along with their type, subcellular components, and connectivity have the potential to reveal fundamental organizational principles of neural circuits. Volumetric nanometer-resolution imaging of brain tissue provides the raw data needed to build such maps, but inferring all the relevant cellular and subcellular annotation layers is challenging. Here, we present Segmentation-Guided Contrastive Learning of Representations ("SegCLR"), a self-supervised machine learning technique that produces highly informative representations of cells directly from 3d electron microscope imagery and segmentations. When applied to volumes of human and mouse cerebral cortex, SegCLR enabled the classification of cellular subcompartments (axon, dendrite, soma, astrocytic process) with 4,000-fold less labeled data compared to fully supervised approaches. Surprisingly, SegCLR also enabled inference of cell types (neuron vs. glia and subtypes of each) from fragments with lengths as small as 10 micrometers, a task that can be difficult for humans to perform and whose feasibility greatly enhances the utility of imaging portions of brains in which many neuron fragments terminate at a volume boundary. These predictions were further augmented via Gaussian process uncertainty estimation to enable analyses restricted to high confidence subsets of the data.

Cell-type-specific inhibitory circuitry from a connectomic census of mouse visual cortex

“…EM imagery and segmentations, as well as skeletons and cell type tables, are publicly available via https://www.microns-explorer.org/cortical-mm3. All analysis was performed in Python 3.9 using custom code, making extensive use of CAVEclient (https://github.com/seung-lab/CAVEclient) and CloudVolume 112 to interact with data infrastructure, MeshParty 113 to analyze skeletons, and libraries Matplotlib 114 , Numpy 115 , Pandas 116 , Scikit-learn 117 , Scipy 118 , stats-models 111 and VTK 119 for general computation, machine learning and data visualization.…”

NEURD offers automated proofreading and feature extraction for connectomics

“…Compared to light microscopy, the ultra-high resolution of EM reconstructions provides many additional features that can be used to classify cells by morphology. Previous studies have demonstrated that rich information enabling cell-type classification is available even in local nuclear and peri-somatic features (Elabbady et al, 2022;Al-Thelaya et al, 2021), small segments of neural processes (Dorkenwald et al, 2022c), and the shape of postsynaptic regions (Seshamani et al, 2020). NEURD provides an additional rich and interpretable feature set that can be used for cell-type classification via a number of different approaches.…”

Synthetic data enable experiments in atomistic machine learning

“…80 We adapt the above recipe by using the Euclidean distance between NN penultimate hidden-layer representations as a dissimilarity metric, and embed the resulting matrix using UMAP, 67 a general approach not limited to chemistry. 83 Figure 6 shows the resulting structure maps. We use 30,000 atomic environments selected at random from the dataset presented above.…”