Metadata and Reproducibility: A Case Study of Gravitational Wave Research Data Management

Abstract

The complexity of computationally-intensive scientific research poses great challenges for both research data management and research reproducibility. What metadata needs to be captured for tracking, reproducing, and reusing computational results is the starting point in developing metadata models to fulfil these functions of data management. This paper reports the findings from interviews with gravitational wave (GW) researchers, which were designed to gather user requirements to develop a metadata model. Motivations for keeping documentation of data and analysis results include trust, accountability and continuity of work. Research reproducibility relies on metadata that represents code dependencies and versions and has good documentation for verification. Metadata specific to GW data, workflows and outputs tend to differ from those currently available in metadata standards. The paper also discusses the challenges in representing code dependencies and workflows..

Open is not enough

“…To foster and enable reproducibility and reuse, scientists must follow comprehensive RDM practices [45,46,154,177]. RDM is referred to as "the organisation of data, from its entry to the research cycle through to the dissemination and archiving of valuable results" [198].…”

The role of metadata in reproducible computational research

“…Review steps included: (1) defining key components of the analytic stack, and functions that metadata can support; (2) selecting exemplary metadata standards that address aspects of the identified functions; (3) assessing the applicability of these standards for supporting computational reproducibility functions; and (4) designing the corresponding metadata hierarchy. Our approach was informed, in part, by the Qin LIGO case study, 46 catalogs of metadata standards such as FAIRSharing, and comprehensive projects to bind semantic science such as Research Objects. 47 Compilation of core materials was accomplished mainly through literature searches but also perusal of code repositories, ontology catalogs, presentations, and Twitter posts.…”

Considerations and Challenges for Describing Historical Research Data: A Case Study

“…Chao's study focused on soil ecology; Ferguson (2012) explains a similar approach for describing biomedical datasets. Finally, in some cases it may be possible and worthwhile to create an entirely new metadata model for a particular dataset (Qin, J., Dobreski, B., & Brown, D., 2016).…”