Toward A Reproducible, Scalable Framework for Processing Large Neuroimaging Datasets

Abstract

Emerging neuroimaging datasets (collected through modalities such as Electron Microscopy, Calcium Imaging, or X-ray Microtomography) describe the location and properties of neurons and their connections at unprecedented scale, promising new ways of understanding the brain. These modern imaging techniques used to interrogate the brain can quickly accumulate gigabytes to petabytes of structural brain imaging data. Unfortunately, many neuroscience laboratories lack the computational expertise or resources to work with datasets of this size: computer vision tools are often not portable or scalable, and there is considerable difficulty in reproducing results or extending methods. We developed an ecosystem of neuroimaging data analysis pipelines that utilize open source algorithms to create standardized modules and end-to-end optimized approaches. As exemplars we apply our tools to estimate synapse-level connectomes from electron microscopy data and cell distributions from X-ray microtomography data. To facilitate scientific discovery, we propose a generalized processing framework, that connects and extends existing open-source projects to provide large-scale data storage, reproducible algorithms, and workflow execution engines. Our accessible methods and pipelines demonstrate that approaches across multiple neuroimaging experiments can be standardized and applied to diverse datasets. The techniques developed are demonstrated on neuroimaging datasets, but may be applied to similar problems in other domains.

Leveraging Tools from Autonomous Navigation for Rapid, Robust Neuron Connectivity

“…For the purposes of the above comparison, these graphs were generated from relatively small volumes. This algorithm can be readily adapted to batch processing frameworks (e.g., [11]) and large spatial databases (e.g., [12]).…”

An Integrated Toolkit for Extensible and Reproducible Neuroscience

“…We designed our toolkit such that even minimal coding skills and copy-pasting of simple design patterns can be leveraged to reduce user burden. As the community continues to formalize use cases and data storage paradigms, programmatic workflows like SABER [ 11 ], LONI [ 12 ], Luigi [ 13 ], or other workflow managers [ 14 ], [ 15 ], [ 16 ] may allow for additional simplification and can directly leverage these functions. Point and click graphical interfaces may also follow.…”

“…4) Processing: Tool and algorithm developers commonly target specific data storage ecosystems in order to reduce the burden of supporting several disparate ecosystems and data-standards [23], [11]. By leveraging shock-absorber tools like intern, algorithm developers can write code once and deploy it to a variety of datastores.…”

intern: Integrated Toolkit for Extensible and Reproducible Neuroscience

“…We designed our toolkit such that even minimal coding skills and copy-pasting of simple design patterns can be leveraged to reduce user burden. As the community continues to formalize use cases and data storage paradigms, programmatic workflows like SABER [12], LONI [13], Luigi [14], or other workflow managers [15,16,17] may allow for additional simplification and can directly leverage these functions. Point and click graphical interfaces may also follow.…”

“…By leveraging shock-absorber tools like intern, algorithm developers can write code once and deploy it to a variety of datastores. As a proof of concept, we adopted a synapse-detection algorithm based upon the U-net architecture [12,24]. This algorithm Service targets data downloaded from an intern VolumeResource, which means that it is trivially portable to data downloaded from any supported volumetric data storage service.…”