On the evaluation of research software: the CDUR procedure

Abstract

Evaluation of the quality of research software is a challenging Background: and relevant issue, still not sufficiently addressed by the scientific community.Our contribution begins by defining, precisely but widely enough, Methods: the notions of research software and of its authors followed by a study of the evaluation issues, as the basis for the proposition of a sound assessment protocol: the CDUR procedure.CDUR comprises four steps introduced as follows: itation, to Results: C deal with correct RS identification, issemination, to measure good D dissemination practices, se, devoted to the evaluation of usability U aspects, and esearch, to assess the impact of the scientific work. R Some conclusions and recommendations are finally Conclusions:included. The evaluation of research is the keystone to boost the evolution of the Open Science policies and practices. It is as well our belief that research software evaluation is a fundamental step to induce better research software practices and, thus, a step towards more efficient science.

TRAAC framework to improve regulatory acceptance and wider usability of tools and methods for safe innovation and sustainability of manufactured nanomaterials

“…The TRAAC framework takes transparency into account by answering three underlying questions ( EC, 2019a ; Gomez-Diaz and Recio, 2019 ; OECD, 2005 ), which lead to the development of four transparency related criteria in Table 1 . Who created the T&M?…”

“…The reliability pillar of the TRAAC framework focuses on aforementioned aspects by assessing the correctness and consistency of the tools' outputs (see Table 2 ). Three underlying questions are addressed ( Aerts, 2017 ; EPA, 2017 ; Gomez-Diaz and Recio, 2019 ; Hristozov et al, 2016 ; Isigonis et al, 2019 ; JRC, 2018 ; Morris et al, 2010 ; OECD, 2005 ; Sørensen et al, 2019 ) when considering the reliability of the T&M, which lead to the development of six reliability related criteria in Table 2 . Has the T&M been verified and received support within the scientific community?…”

The Research Software Encyclopedia: A Community Framework to Define Research Software

“…This kind of context is equally important when we discuss research software, as the needs of a group or individual clearly frame any subsequent evaluation. Although efforts such as FAIR [1] exist to ensure that software in the research domain is findable, accessible, interoperable, and reusable (FAIR), and there is work to define the life-cycle [2] or measuring of such software [7], these efforts focus on quality or best practices, which is a different task than definition. There is also often an implied bias that the definition is self explanatory, and that research software is simply software that is used in research [3,6].…”

Research Software vs. Research Data II: Protocols for Research Data dissemination and evaluation in the Open Science context

“…Is it good enough to use metadata purely to discover the functionality of a RS code, or might it be necessary to explore in more details the functions and libraries in RS? It is maybe a naïve vision, but research data features a much broader heterogeneity than RS, so many different types of data can be generated that it makes these procedures difficult to apply or generalise to RD in general (As mentioned in the report of reviewer 1 for RS 2 ). In general, more specific examples are provided for RS than for RD.…”

“…It is maybe a naïve vision, but research data features a much broader heterogeneity than RS, so many different types of data can be generated that it makes these procedures difficult to apply or generalise to RD in general (As mentioned in the report of reviewer 1 for RS 2 ).…”