Large-scale automatic reconstruction of neuronal processes from electron microscopy images

Abstract

Automated sample preparation and electron microscopy enables acquisition of very large image data sets. These technical advances are of special importance to the field of neuroanatomy, as 3D reconstructions of neuronal processes at the nm scale can provide new insight into the fine grained structure of the brain. Segmentation of large-scale electron microscopy data is the main bottleneck in the analysis of these data sets. In this paper we present a pipeline that provides state-of-the art reconstruction performance while scaling to data sets in the GB-TB range. First, we train a random forest classifier on interactive sparse user annotations. The classifier output is combined with an anisotropic smoothing prior in a Conditional Random Field framework to generate multiple segmentation hypotheses per image. These segmentations are then combined into geometrically consistent 3D objects by segmentation fusion. We provide qualitative and quantitative evaluation of the automatic segmentation and demonstrate large-scale 3D reconstructions of neuronal processes from a 27,000 μm3 volume of brain tissue over a cube of 30 μm in each dimension corresponding to 1,000 consecutive image sections. We also introduce Mojo, a proofreading tool including semi-automated correction of merge errors based on sparse user scribbles.

Structured learning of assignment models for neuron reconstruction to minimize topological errors

“…Assignment models for anisotropic neuron reconstruction introduce n binary indicator variables z ∈ {0, 1} n to represent all possible assignments of 2D neuron candidates across all pairs of sections of a volume [1,2]. Linear constraints are formulated on the binary assignment indicators to ensure that a solution is consistent, i.e., no pair of overlapping candidates is selected.…”

“…In this paper, we present a structured learning framework to train assignment models [1,2] for anisotropic neuron reconstruction. Our contributions are: (1) We show how to generate a training sample suitable for structured learning from human annotated ground truth.…”

“…Our contributions are: (1) We show how to generate a training sample suitable for structured learning from human annotated ground truth. (2) We introduce a loss for structured learning, which minimizes topological errors during learning. (3) We show how our learning framework can be used to train on skeleton (i.e., non-volumetric) annotations, which are in practice much faster to obtain.…”

“…Related Work. Assignment models constitute the current state of the art for the reconstruction of neurons from anisotropic volumes, as obtained by serial section EM [1,2]. These models enumerate and price possible assignments of candidate segments across sections of EM stacks (see Figure 1 for an overview and Section 2 for details).…”

“…Currently, these costs are set by hand [3,2,4], learned from a random forest classifier based on positive and negative assignment examples [1,5], or found via grid-search on linear weights for a small number of features [6]. Except for grid-search, which does not scale to larger sets of parameters, none of the currently used training methods implements real end-to-end learning.…”