Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction

Abstract

We present a method combining affinity prediction with region agglomeration, which improves significantly upon the state of the art of neuron segmentation from electron microscopy (EM) in accuracy and scalability. Our method consists of a 3D U-net, trained to predict affinities between voxels, followed by iterative region agglomeration. We train using a structured loss based on MALIS, encouraging topologically correct segmentations obtained from affinity thresholding. Our extension consists of two parts: First, we present a quasi-linear method to compute the loss gradient, improving over the original quadratic algorithm. Second, we compute the gradient in two separate passes to avoid spurious gradient contributions in early training stages. Our predictions are accurate enough that simple learning-free percentile-based agglomeration outperforms more involved methods used earlier on inferior predictions. We present results on three diverse EM datasets, achieving relative improvements over previous results of 27%, 15%, and 250%. Our findings suggest that a single method can be applied to both nearly isotropic block-face EM data and anisotropic serial sectioned EM data. The runtime of our method scales linearly with the size of the volume and achieves a throughput of ~2.6 seconds per megavoxel, qualifying our method for the processing of very large datasets.

Binary and analog variation of synapses between cortical pyramidal neurons

“…The watershed supervoxels and affinity map were divided into 513×513×129 chunks that overlapped by 1 in each direction. Each chunk was processed using mean affinity agglomeration (Lee et al , 2017;Funke et al , 2019) . Agglomeration decisions at chunk boundaries were delayed, and information about the decisions was saved to disk.…”

Accurate and Versatile 3D Segmentation of Plant Tissues at Cellular Resolution

“…In the second step, the final segmentation is computed as a partitioning of this graph into an unknown number of objects (see Figure 1). Our choice of graph partitioning as the second step is inspired by a body of work on segmentation for nanoscale connectomics (segmentation of cells in electron microscopy images of neural tissue), where such methods have been shown to outperform more simple post-processing of the boundary maps [13,15,25].…”

“…Once the boundaries are found, other pixels need to be grouped into objects delineated by the detected boundaries. For noisy, real-world microscopy data, this post-processing step still represents a challenge and has attracted a fair amount of attention from the computer vision community [11,12,13,14,15]. If centroids ("seeds") of the objects are known or can be learned, the problem can be solved by the watershed algorithm [16,17].…”

“…PlantSeg processes the output from the microscope with a CNN to produce an accurate prediction of cell boundaries. Building on the insights from previous work on cell segmentation in electron microscopy volumes of neural tissue [13,15], the second step of the pipeline delivers an accurate segmentation by solving a graph partitioning problem. We trained PlantSeg on 3D confocal images of fixed Arabidopsis thaliana ovules and 3D+t light sheet microscope images of developing lateral roots, two standard imaging modalities in the studies of plant morphogenesis.…”

Automatic Detection of Synaptic Partners in a Whole-BrainDrosophilaEM Dataset

“…We use a 3D U-Net (Falk et al, 2019) as our core architecture, following the design used by Funke et al (2018) and Heinrich et al (2018), i.e., we use four resolution levels with downsample factors in xyz of (3, 3, 1), (3, 3, 1), and (3, 3, 3) and a five-fold feature map increase between levels. Convolutional passes are comprised of two convolutions with kernel sizes of (3, 3, 3) followed by a R LU activation.…”