The open connectome project data cluster

Abstract

We describe a scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data, initially for 3-d electron microscopy image stacks, but for time-series and multi-channel data as well. The system was designed primarily for workloads that build connectomes— neural connectivity maps of the brain—using the parallel execution of computer vision algorithms on high-performance compute clusters. These services and open-science data sets are publicly available at openconnecto.me. The system design inherits much from NoSQL scale-out and data-intensive computing architectures. We distribute data to cluster nodes by partitioning a spatial index. We direct I/O to different systems—reads to parallel disk arrays and writes to solid-state storage—to avoid I/O interference and maximize throughput. All programming interfaces are RESTful Web services, which are simple and stateless, improving scalability and usability. We include a performance evaluation of the production system, highlighting the effec-tiveness of spatial data organization.

An automated images-to-graphs framework for high resolution connectomics

“…Our approach for creating brain graphs from large image volumes (images-to-graphs) leverages the Open Connectome Project ( www.openconnecto.me ) infrastructure (Burns et al, 2013 ).…”

“…This webserver distributes jobs across three data servers running a distributed database (each with 2x Intel Xeon X5690 3.47 GHz 12 core/processor and 48 GB of RAM). Additionally, 100 TB of network mounted disk-space is available for storage (Burns et al, 2013 ). An overall schematic of our infrastructure is shown in Figure 2 .…”

“…Therefore, we developed our own lightweight, scalable synapse detector. Finally, the Open Connectome Project (Burns et al, 2013 ) provides a high-performance spatial database optimized for processing large neuroimaging volumes. These tools facilitate scalable processing and provide significant advances over a flat-file architecture in terms of data storage and access.…”

Knowing a synapse when you see one

“…Even “simple” neocortical networks such as the well-studied rodent whisker column comprise hundreds of millions of densely-packed synapses, precluding the satisfactory application of traditional labor-intensive approaches to single-synapse analysis. Emerging methods of computational microscopy (Burns et al, 2013 ) are now poised to advance single-synapse imaging and measurement to the necessary scale, so careful thought is now due to questions about what we should be looking for, and what we need to measure once we see it.…”

High-Resolution Synaptic Connectomics

“…all build becomes globally available via Viking-like Web tools for annotation and analysis a network of existing TeMs based on SerialeM acquisition, our new open source Nornir build system (available at github), and Viking annotation, can accelerate connectome assembly and use by over 100-fold. indeed, new initiatives, such as the open connectome Project [20], inspire the idea that significant advances in analytical speed can be had as well. in contrast, single system enhancements via faster cameras and more stable scanning stages may yield improvements 2-3 ×.…”