A Link is not Enough – Reproducibility of Data

Pawlik, Mateusz; Hütter, Thomas; Kocher, Daniel; Mann, Willi; Augsten, Nikolaus

doi:10.1007/s13222-019-00317-8

Cited by 16 publications

(19 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We assign meaning to the term as data as an output (result) in a computational context, which was generated when reproducing computational experiments. Although the term “computationally reproducible data” is not officially defined, other sources and studies have referred to the concept of data that contributes to computational reproducibility ( Baranyi and Greilhuber, 1999 ; Weinländer et al, 2009 ; de Ruiter, 2017 ; Perkel, 2017 ; Tait, 2017 ; Pawlik et al, 2019 ). From the unsure responses ( n = 30), we categorised those that gave free-text responses (70%, n = 21, see Supplementary section 2, free responses) into whether they did actually understand the term, those that did not understand the term, and those that did not give any free text.…”

Section: Resultsmentioning

confidence: 99%

Knowledge and Attitudes Among Life Scientists Toward Reproducibility Within Journal Articles: A Research Survey

Samota

Davey

2021

Front. Res. Metr. Anal.

View full text Add to dashboard Cite

We constructed a survey to understand how authors and scientists view the issues around reproducibility, focusing on interactive elements such as interactive figures embedded within online publications, as a solution for enabling the reproducibility of experiments. We report the views of 251 researchers, comprising authors who have published in eLIFE Sciences, and those who work at the Norwich Biosciences Institutes (NBI). The survey also outlines to what extent researchers are occupied with reproducing experiments themselves. Currently, there is an increasing range of tools that attempt to address the production of reproducible research by making code, data, and analyses available to the community for reuse. We wanted to collect information about attitudes around the consumer end of the spectrum, where life scientists interact with research outputs to interpret scientific results. Static plots and figures within articles are a central part of this interpretation, and therefore we asked respondents to consider various features for an interactive figure within a research article that would allow them to better understand and reproduce a published analysis. The majority (91%) of respondents reported that when authors describe their research methodology (methods and analyses) in detail, published research can become more reproducible. The respondents believe that having interactive figures in published papers is a beneficial element to themselves, the papers they read as well as to their readers. Whilst interactive figures are one potential solution for consuming the results of research more effectively to enable reproducibility, we also review the equally pressing technical and cultural demands on researchers that need to be addressed to achieve greater success in reproducibility in the life sciences.

show abstract

Section: Resultsmentioning

confidence: 99%

Knowledge and Attitudes Among Life Scientists Toward Reproducibility Within Journal Articles: A Research Survey

Samota

Davey

2021

Front. Res. Metr. Anal.

View full text Add to dashboard Cite

show abstract

“…Even with current policies mandating data openness [59,60], authors still fail to include their data alongside their publication, and this can not only be attributed to technical complications, but also fear of being scooped, fear of mistakes being found in data or analyses, and fear of others using their data for their own research papers [64,73]. Making FAIR data practices standard, through public data deposition and subsequent publication and citation, could encourage individual researchers and communities to share and reus e data considering their individual requirements and needs [68]. Data accessibility issues are also compounded by data becoming less retrievable with every year passing after the publication [85].…”

Section: Discussionmentioning

confidence: 99%

“…This raises complex questions around large data generation projects that also need to be studied extensively for future impact, especially with respect to reproducibility within publications. Moreover, access to the raw data might not be enough, if the steps and other artefacts involved in producing the processed data that was used in the analysis are not provided [68]. In addition, corresponding authors often move on from projects and institutions or the authors themselves can no longer access the data, meaning "data available on request" ceases to be a viable option to source data or explanations of methods.…”

Section: Discussionmentioning

confidence: 99%

“…We assign meaning to the term as data as an output (result) in a computational context, which was generated when reproducing computational experiments. Although the term "computationally reproducible data" is not officially defined , other sources and studies have referred to the concept of data that contributes to computational reproducibility [26,[66][67][68][69][70]. From the unsure responses (n=30), we categorised those that gave free-text responses (70%, n=21, see Supplementary section 2, free responses) into whether they did actually understand the term, those that did not understand the term, and those that did not give any free text.…”

Section: Understanding Of Reproducibility Training and Successful Rementioning

confidence: 99%

See 1 more Smart Citation

Knowledge and attitudes among life scientists towards reproducibility within journal articles: a research survey

Samota

Davey

2019

Preprint

View full text Add to dashboard Cite

We constructed a survey to understand how authors and scientists view the issues around reproducibility, and how solutions such as interactive figures could enable the reproducibility of experiments from within a research article. This manuscript reports the results of this survey on the views of 251 researchers, including authors who have published in eLIFE Sciences, and those who work at the Norwich Biosciences Institutes (NBI). The survey also outlines to what extent researchers are occupied with reproducing experiments themselves and what are their desirable features of an interactive figure. Respondents considered various features for an interactive figure within a research article that would allow for them to better understand and reproduce in situ the experiment presented in the figure. Respondents said that the most important element that would enable the better reproducibility of published research would be that authors describe methods and analyses in detail.The respondents believe that having an interactive figures in published papers is a beneficial element.Whilst interactive figures are potential solutions for demonstrating technical reproducibility, we find that there are equally pressing cultural demands on researchers that need to be addressed to achieve greater success in reproducibility in the life sciences .

show abstract

“…But why is it so important for reproducibility and archival access to access materials in the original computational environment? Given how much modern research practices rely on unique toolkits, the output from any analysis is highly dependent on the actual software in which the research happens (Pawlik et al, 2019). Results of research change depending on versions of software, on operating system changes, and other custom configurations (Gronenschild et al, 2012).…”

Section: Introductionmentioning

confidence: 99%

Reproducibility, preservation, and access to research with ReproZip and ReproServer

Steeves

Rampin

Chirigati

2020

View full text Add to dashboard Cite

The adoption of reproducibility remains low, despite incentives becoming increasingly common in different domains, conferences, and journals. The truth is, reproducibility is technically difficult to achieve due to the complexities of computational environments. To address these technical challenges, we created ReproZip, an open-source tool that automatically packs research along with all the necessary information to reproduce it, including data files, software, OS version, and environment variables. Everything is then bundled into an rpz file, which users can use to reproduce the work with ReproZip and a suitable unpacker (e.g.: using Vagrant or Docker). The rpz file is general and contains rich metadata: more unpackers can be added as needed, better guaranteeing long-term preservation. However, installing the unpackers can still be burdensome for secondary users of ReproZip bundles. In this paper, we will discuss how ReproZip and our new tool, ReproServer, can be used together to facilitate access to well-preserved, reproducible work. ReproServer is a web application that allows users to upload or provide a link to a ReproZip bundle, and then interact with/reproduce the contents from the comfort of their browser. Users are then provided a persistent link to the unpacked work on ReproServer which they can share with reviewers or colleagues.

show abstract

A Link is not Enough – Reproducibility of Data

Cited by 16 publications

References 18 publications

Knowledge and Attitudes Among Life Scientists Toward Reproducibility Within Journal Articles: A Research Survey

Knowledge and Attitudes Among Life Scientists Toward Reproducibility Within Journal Articles: A Research Survey

Knowledge and attitudes among life scientists towards reproducibility within journal articles: a research survey

Reproducibility, preservation, and access to research with ReproZip and ReproServer

Contact Info

Product

Resources

About