Open Science Expectations for Simulation-Based Research

Abstract

There is strong agreement across the sciences that replicable workflows are needed for computational modeling. Open and replicable workflows not only strengthen public confidence in the sciences, but also result in more efficient community science. However, the massive size and complexity of geoscience simulation outputs, as well as the large cost to produce and preserve these outputs, present problems related to data storage, preservation, duplication, and replication. The simulation workflows themselves present additional challenges related to usability, understandability, documentation, and citation. These challenges make it difficult for researchers to meet the bewildering variety of data management requirements and recommendations across research funders and scientific journals. This paper introduces initial outcomes and emerging themes from the EarthCube Research Coordination Network project titled “What About Model Data? - Best Practices for Preservation and Replicability,” which is working to develop tools to assist researchers in determining what elements of geoscience modeling research should be preserved and shared to meet evolving community open science expectations.Specifically, the paper offers approaches to address the following key questions:• How should preservation of model software and outputs differ for projects that are oriented toward knowledge production vs. projects oriented toward data production?• What components of dynamical geoscience modeling research should be preserved and shared?• What curation support is needed to enable sharing and preservation for geoscience simulation models and their output?• What cultural barriers impede geoscience modelers from making progress on these topics?

Understanding the value of curation: A survey of researcher perspectives of data curation services from six US institutions

“…Many of these impediments could be mitigated by working with a curator who offers recommendations to improve the dataset before its publication. As Mullendore et al note, “Curation tasks should involve partnerships between scientific and data experts to take advantage of their respective knowledge and skills” [ 18 ].…”

“…Researchers wrestle with documentation and metadata in their own work due to lack of access to standards and templates, which is a drain on resources like time and labor [ 4 , 21 ]. Mullendore et al reported that some researchers spend significant amounts of their time curating their data, sometimes “over 50% of their funded time” [ 18 ]. When Johnston et al asked researchers what curators could do to help them in their work, overwhelmingly they responded with anything they could hand off to a curator so the researcher didn’t have to deal with it [ 3 ].…”