Comment on “Most computational hydrology is not reproducible, so is it really science?” by Christopher Hutton et al.

Abstract

Nowadays, the majority of the scientific community is not aware of the risks and problems associated with an inadequate use of computer systems for research, mostly for reproducibility of scientific results. Such reproducibility can be compromised by the lack of clear standards and insufficient methodological description of the computational details involved in an experiment. In addition, the inappropriate application or ignorance of copyright laws can have undesirable effects on access to aspects of great importance of the design of experiments and therefore to the interpretation of results.

Reply to comment by Añel on “Most computational hydrology is not reproducible, so is it really science?”

“…We agree with Añel [2017] on the importance of version control and on clear notification of hardware used. These will likely be included as metadata fields for code and workflows, as we discuss the need for in our commentary [Hutton et al, 2016], which need to be evolved for hydrological code.…”

“…Therefore, we urge for an attitude shift, including by ourselves and our colleagues. As mentioned by Añel [2017], it is also a problem with publishers, editors, and reviewers; the problem is recognized but not seriously dealt with yet.In hydrological sciences, many assumptions are made as there are many unknown boundary conditions and poorly determined properties in the system. Arguably more than in areas such as physiology and pharmacology, where the reproducibility issue is already well discussed (see citations in Hutton et al [2016]).…”

“…However, we believe that from changing the culture and attitude among hydrological scientists, details will evolve to cover more (technical) aspects over time. Here we deal with these technical issues raised by Añel [2017], in turn.We agree with Añel [2017] on the importance of version control and on clear notification of hardware used. These will likely be included as metadata fields for code and workflows, as we discuss the need for in our commentary [Hutton et al, 2016], which need to be evolved for hydrological code.…”

“…However, in our view, it is the cultural change that is the greatest challenge to overcome to achieve reproducible scientific research in computational hydrology. We believe that from changing the culture and attitude among hydrological scientists, details will evolve to cover more (technical) aspects over time.We would first like to thank Añel [2017] for responding to our commentary paper [Hutton et al, 2016], and contributing some important additional points to the reproducibility issues we have raised. We fully agree that the problematic issue with reproducibility is not new, neither to science nor to computing technology, and we explicitly acknowledge this in the commentary as problem more broadly in the scientific literature [Hutton et al, 2016].…”

“…We would first like to thank Añel [2017] for responding to our commentary paper [Hutton et al, 2016], and contributing some important additional points to the reproducibility issues we have raised. We fully agree that the problematic issue with reproducibility is not new, neither to science nor to computing technology, and we explicitly acknowledge this in the commentary as problem more broadly in the scientific literature [Hutton et al, 2016].…”