Neurodata Without Borders: Creating a Common Data Format for Neurophysiology

Teeters, Jeffery L.; Godfrey, Keith B.; Young, Rob; Dang, Chinh; Friedsam, Claudia; Wark, Barry; Asari, Hiroki; Peron, Simon; Li, Nuo; Peyrache, Adrien; Denisov, Gennady; Siegle, Joshua H.; Olsen, Shawn R.; Martin, Christopher L.; Chun, Miyoung; Tripathy, Shreejoy J.; Blanche, Timothy J.; Harris, Kenneth D.; Buzsáki, György; Koch, Christof; Meister, Markus; Svoboda, Karel; Sommer, Friedrich T.

doi:10.1016/j.neuron.2015.10.025

Cited by 170 publications

(175 citation statements)

References 24 publications

Supporting

Mentioning

167

Contrasting

Order By: Relevance

“…An alternative data storage approach is to reuse experimental data storage formats, although the experimental community also lacks standards. The Neurodata Without Borders framework for data sharing is a collaborative project involving several large research groups [143]. NIX (GitHub.com/G-Node/nix) is an alternative format under development by the INCF German Node.…”

Section: Repositories and Standards: Code Data And Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Reproducibility in Computational Neuroscience Models and Simulations

McDougal

Bulanova

Lytton

2016

IEEE Trans. Biomed. Eng.

View full text Add to dashboard Cite

Objective Like all scientific research, computational neuroscience research must be reproducible. Big data science, including simulation research, cannot depend exclusively on journal articles as the method to provide the sharing and transparency required for reproducibility. Methods Ensuring model reproducibility requires the use of multiple standard software practices and tools, including version control, strong commenting and documentation, and code modularity. Results Building on these standard practices, model sharing sites and tools have been developed that fit into several categories: 1. standardized neural simulators, 2. shared computational resources, 3. declarative model descriptors, ontologies and standardized annotations; 4. model sharing repositories and sharing standards. Conclusion A number of complementary innovations have been proposed to enhance sharing, transparency and reproducibility. The individual user can be encouraged to make use of version control, commenting, documentation and modularity in development of models. The community can help by requiring model sharing as a condition of publication and funding. Significance Model management will become increasingly important as multiscale models become larger, more detailed and correspondingly more difficult to manage by any single investigator or single laboratory. Additional big data management complexity will come as the models become more useful in interpreting experiments, thus increasing the need to ensure clear alignment between modeling data, both parameters and results, and experiment.

show abstract

Section: Repositories and Standards: Code Data And Modelsmentioning

confidence: 99%

“…Additional specific efforts for experimental data sharing include Collaborative Research in Computational Neuroscience [141] (crcns.org) hosting Neurodata Without Borders [143], Hippocampome [160] (hippocampome.org), and NeuroElectro [161] (neuroelectro.org). …”

Section: Broader Issues – Experimental Data Sharing and Model Matmentioning

confidence: 99%

Reproducibility in Computational Neuroscience Models and Simulations

McDougal

Bulanova

Lytton

2016

IEEE Trans. Biomed. Eng.

View full text Add to dashboard Cite

show abstract

“…Some of these formats are discussed by Teeters et al (2015) where they also introduce Neurodata Without Borders (NWB), a format recently developed in an attempt to unify cellularbased neurophysiology data and break down barriers for data sharing. Many of these formats, including NWB, are based on HDF5 and therefore share the same advantages and disadvantages as HDF5.…”

Section: Commonly Used Open Formats In Neurosciencementioning

confidence: 99%

“…HDF5 is the basis of many formats in neuroscience, including the recent collaborative format, Neurodata Without Borders (NWB) (Teeters et al, 2015). However, issues with HDF5 have recently surfaced in the neuroscience community (Rossant, 2016b).…”

Section: Introductionmentioning

confidence: 99%

Experimental Directory Structure (Exdir): An alternative to HDF5 without introducing a new file format

Dragly

Mobarhan

Lepperød

et al. 2018

Preprint

View full text Add to dashboard Cite

Natural sciences generate an increasing amount of data in a wide range of formats developed by different research groups and commercial companies. At the same time there is a growing desire to share data along with publications in order to enable reproducible research. Open formats have publicly available specifications which facilitate data sharing and reproducible research. Hierarchical Data Format 5 (HDF5) is a popular open format widely used in neuroscience, often as a foundation for other, more specialized formats. However, drawbacks related to HDF5's complex specification have initiated a discussion for an improved replacement. We propose a novel alternative, the Experimental Directory Structure (Exdir), an open standard for data storage in experimental pipelines which amends drawbacks associated with HDF5 while retaining its advantages. HDF5 stores data and metadata in a hierarchy within a complex binary file which, among other things, is not human-readable, not optimal for version control systems, and lacks support for storing raw data. Exdir, one the other hand, uses file system directories to represent the hierarchy, with metadata stored in human-readable YAML files, datasets stored in binary NumPy files, and raw data stored directly in subdirectories. Furthermore, storing data in multiple files makes it easier to track for version control systems. Exdir is not a file format in itself, but a standard for organizing files in a directory structure. Exdir uses the same abstractions as HDF5 and is compatible with the HDF5 Abstract Data Model. Several research groups are already using data stored in a directory hierarchy as an alternative to HDF5, but no common standard exists in the field. This complicates and limits the opportunity for data sharing and development of common tools for reading, writing, and analyzing data. Exdir facilitates improved data storage, data sharing, reproducible research, and novel insight from interdisciplinary collaboration. With the publication of Exdir, we invite the scientific community to join the development to create an open standard that will serve as many needs as possible and that will serve as a foundation for open access to and exchange of data.

show abstract

“…NeuroML (Gleeson et al, 2010) is also an XML-based format with a particular focus on defining and exchanging descriptions of neuronal cell and network models. The Neurodata Without Borders (NWB) (Teeters et al, 2015) initiative is a recent project with the specific goal “ […] to produce a unified data format for cellular-based neurophysiology data based on representative use cases initially from four laboratories—the Buzsaki group at NYU, the Svoboda group at Janelia Farm, the Meister group at Caltech, and the Allen Institute for Brain Science in Seattle.” Members of the NIX, KWIK, Orca, BRAINformat, and other development teams have been invited and contributed to the NWB effort. NWB has adopted concepts and methods from a range of these formats, including from the here-described BRAINformat.…”

Section: Background and Related Workmentioning

confidence: 99%

Methods for Specifying Scientific Data Standards and Modeling Relationships with Applications to Neuroscience

Rübel

Dougherty

Prabhat

et al. 2016

Front. Neuroinform.

View full text Add to dashboard Cite

Neuroscience continues to experience a tremendous growth in data; in terms of the volume and variety of data, the velocity at which data is acquired, and in turn the veracity of data. These challenges are a serious impediment to sharing of data, analyses, and tools within and across labs. Here, we introduce BRAINformat, a novel data standardization framework for the design and management of scientific data formats. The BRAINformat library defines application-independent design concepts and modules that together create a general framework for standardization of scientific data. We describe the formal specification of scientific data standards, which facilitates sharing and verification of data and formats. We introduce the concept of Managed Objects, enabling semantic components of data formats to be specified as self-contained units, supporting modular and reusable design of data format components and file storage. We also introduce the novel concept of Relationship Attributes for modeling and use of semantic relationships between data objects. Based on these concepts we demonstrate the application of our framework to design and implement a standard format for electrophysiology data and show how data standardization and relationship-modeling facilitate data analysis and sharing. The format uses HDF5, enabling portable, scalable, and self-describing data storage and integration with modern high-performance computing for data-driven discovery. The BRAINformat library is open source, easy-to-use, and provides detailed user and developer documentation and is freely available at: https://bitbucket.org/oruebel/brainformat.

show abstract

Neurodata Without Borders: Creating a Common Data Format for Neurophysiology

Cited by 170 publications

References 24 publications

Reproducibility in Computational Neuroscience Models and Simulations

Reproducibility in Computational Neuroscience Models and Simulations

Experimental Directory Structure (Exdir): An alternative to HDF5 without introducing a new file format

Methods for Specifying Scientific Data Standards and Modeling Relationships with Applications to Neuroscience

Contact Info

Product

Resources

About