Sustainable computational science: the ReScience initiative

Abstract

Computer science offers a large set of tools for prototyping, writing, running, testing, validating, sharing and reproducing results; however, computational science lags behind. In the best case, authors may provide their source code as a compressed archive and they may feel confident their research is reproducible. But this is not exactly true. James Buckheit and David Donoho proposed more than two decades ago that an article about computational results is advertising, not scholarship. The actual scholarship is the full software environment, code, and data that produced the result. This implies new workflows, in particular in peer-reviews. Existing journals have been slow to adapt: source codes are rarely requested and are hardly ever actually executed to check that they produce the results advertised in the article. ReScience is a peer-reviewed journal that targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research can be replicated from its description. To achieve this goal, the whole publishing chain is radically different from other traditional scientific journals. ReScience resides on GitHub where each new implementation of a computational study is made available together with comments, explanations, and software tests.

Rinse and Repeat: Understanding the Value of Replication across Different Ways of Knowing

“…For instance, Claerbout and Karrenbach [14] define reproduction as the act of "running the same software on the same input data and obtaining the same results" and argue that it may even be automated (see also [15]). Rougier defines replication as "running new software [...] and obtaining results that are similar enough" [15]. The difference between reproduction and replication here lies in the code used, not the data.…”

State of the Art and Future Trends in Data Reduction for High-Performance Computing

“…The ReScience initiative has launched a new peer-reviewed journal that tries to tackle replication problems by using a new publication approach [124]. The whole review and publication process is hosted on GitHub and anyone with a GitHub account is able to comment on a submitted publication.…”

Neuroinformatics and Computational Modelling as Complementary Tools for Neurotoxicology Studies

“…Currently, the source codes written in different programming languages are only limitedly available in model repositories . Therefore, the description of in silico models, their source codes and experimental data used to construct and validate the in silico models should be openly available to the scientific community . The process of combining models to build more comprehensive ones will become significantly easier and make the use of neuroscience in silico models more efficient.…”