Design and Evaluation of Interactive Proofreading Tools for Connectomics

Abstract

Fig. 1: Proofreading with Dojo.We present a web-based application for interactive proofreading of automatic segmentations of connectome data acquired via electron microscopy. Split, merge and adjust functionality enables multiple users to correct the labeling of neurons in a collaborative fashion. Color-coded structures can be explored in 2D and 3D.Abstract-Proofreading refers to the manual correction of automatic segmentations of image data. In connectomics, electron microscopy data is acquired at nanometer-scale resolution and results in very large image volumes of brain tissue that require fully automatic segmentation algorithms to identify cell boundaries. However, these algorithms require hundreds of corrections per cubic micron of tissue. Even though this task is time consuming, it is fairly easy for humans to perform corrections through splitting, merging, and adjusting segments during proofreading. In this paper we present the design and implementation of Mojo, a fully-featured singleuser desktop application for proofreading, and Dojo, a multi-user web-based application for collaborative proofreading. We evaluate the accuracy and speed of Mojo, Dojo, and Raveler, a proofreading tool from Janelia Farm, through a quantitative user study. We designed a between-subjects experiment and asked non-experts to proofread neurons in a publicly available connectomics dataset. Our results show a significant improvement of corrections using web-based Dojo, when given the same amount of time. In addition, all participants using Dojo reported better usability. We discuss our findings and provide an analysis of requirements for designing visual proofreading software.

Scalable Interactive Visualization for Connectomics

“…Proofreading: All available segmentation methods make mistakes [10], and so the results must be proofread by humans before any biological analysis occurs [11][12][13]. For proofreading, intelligent interactive visualization tools are key to minimizing the time committed.…”

“…Scalable Editing: Beyond simply providing data, the proofreading stage of the connectomics workflow also requires interactive data editing [11]. Dynamic data structures such as lookup-tables enable non-destructive editing by storing an edit decision list separately from the actual image data, which then informs the rendering process.…”

“…), a Web-based proofreading application that supports multiple users. We evaluated Dojo and other proofreading software with novices trained by experts as part of a quantitative user study (between-subjects experiment) [11], and designed mechanisms for quality control [13].…”

NeuroLines: A Subway Map Metaphor for Visualizing Nanoscale Neuronal Connectivity

“…In computer science, connectomics has stimulated a lot of research in image processing, vision, and visualization. Most software for connectomics focuses on manual annotation [2,19,38] and (semi-)automatic segmentation [1,21,24] of neuronal structures as well as proof-reading automatic segmentation results [18,37]. However, all of these approaches do not support advanced exploration or visual analysis of features.…”

“…In the next step, our collaborators segment and label this 3D volume, using both manual and automatic segmentation tools [24]. Mojo [25] and Dojo [18] are used for proof-reading automatically generated segmentations. All additional data, such as the locations and properties of synapses, are currently annotated manually and stored as meta data.…”