What does research reproducibility mean?

Goodman, Steven N.; Fanelli, Daniele; Ioannidis, John P. A.

doi:10.1126/scitranslmed.aaf5027

Cited by 921 publications

(789 citation statements)

References 38 publications

Supporting

Mentioning

780

Contrasting

Unclassified

Order By: Relevance

“…A special mention should be made to the expression of the general reproducibility problem in science (Collins & Tabak, 2014) to tDCS studies of motor learning. There are three levels of reproducibility: methods, results and inferential (Goodman, Fanelli, & Ioannidis, 2016). Methodological reproducibility requires "provision of enough detail about study procedures and data so the same procedures could …be exactly repeated".…”

Section: Caveats and Considerations For The Futurementioning

confidence: 99%

“…Reproducibility of results refers to replicability once the tools for methodological replication are fully provided and agreed upon. Importantly, replicability is best tested for stochastic data using Bayesian paradigms of accumulating evidence more than binary criteria of successful or unsuccessful replication (Goodman et al, 2016). Clearly, "statistical significance by itself tells very little about whether one study has "replicated" the results of another".…”

Section: Caveats and Considerations For The Futurementioning

confidence: 99%

“…Please, see Goodman and Ioannidis for a full discussion. (Goodman et al, 2016) A unique concern that has emerged with transcranial electrical stimulation techniques, is that the simplicity, low-cost nature of, and public access to the technology has lead to the emergence of a popular do-it-yourself movement where individuals participate in selfexperimentation without oversight. Such data is not part of well-designed experimental protocols (Fitz & Reiner, 2015;Riggall et al, 2015;Wexler, 2016).…”

Section: Caveats and Considerations For The Futurementioning

confidence: 99%

See 2 more Smart Citations

Effects of tDCS on motor learning and memory formation: A consensus and critical position paper

Buch

Santarnecchi

Antal

et al. 2017

Clinical Neurophysiology

280

231

View full text Add to dashboard Cite

Motor skills are required for activities of daily living. Transcranial direct current stimulation (tDCS) applied in association with motor skill learning has been investigated as a tool for enhancing training effects in health and disease. Here, we review the published literature investigating whether tDCS can facilitate the acquisition and retention of motor skills and adaptation. A majority of reports focused on the application of anodal tDCS over the primary motor cortex (M1) during motor skill acquisition, while some evaluated tDCS applied over the cerebellum during adaptation of existing motor skills. Work in multiple laboratories is under way to develop a mechanistic understanding of tDCS effects on different forms of learning, and to optimize stimulation protocols.Efforts are required to improve reproducibility and standardization. Overall, reproducibility remains to be fully tested, effect sizes with present techniques are moderate (up to d= 0.5) (Hashemirad, Zoghi, Fitzgerald, & Jaberzadeh, 2016) and the basis of inter-individual variability in tDCS effects is incompletely understood. It is recommended that future studies explicitly state in the Methods the exploratory (hypothesisgenerating) or hypothesis-driven (confirmatory) nature of the experimental designs. General research practices could be improved with prospective pre-registration of hypothesis-based investigations, more emphasis on detailed description of methods and use of post-publication open data repositories. A checklist is proposed for reporting tDCS investigations in a way that can improve efforts to assess reproducibility.

show abstract

Section: Caveats and Considerations For The Futurementioning

confidence: 99%

Section: Caveats and Considerations For The Futurementioning

confidence: 99%

Section: Caveats and Considerations For The Futurementioning

confidence: 99%

See 1 more Smart Citation

Effects of tDCS on motor learning and memory formation: A consensus and critical position paper

Buch

Santarnecchi

Antal

et al. 2017

Clinical Neurophysiology

280

231

View full text Add to dashboard Cite

show abstract

“…Despite the success of multilayer network modeling and analysis in systems biology and systems medicine, some methodological challenges are still to be tackled to build consistent, replicable and reproducible [34,35,36,37] representations of multi-omics, connectomics and intercellular interactions as the ones central to the review of Gosak et al [1].…”

Section: Open Challengesmentioning

confidence: 99%

Multilayer network modeling of integrated biological systems

Domenico

2018

Physics of Life Reviews

View full text Add to dashboard Cite

Biological systems, from a cell to the human brain, are inherently complex. A powerful representation of such systems, described by an intricate web of relationships across multiple scales, is provided by complex networks. Recently, several studies are highlighting how simple networks -obtained by aggregating or neglecting temporal or categorical description of biological data -are not able to account for the richness of information characterizing biological systems. More complex models, namely multilayer networks, are needed to account for interdependencies, often varying across time, of biological interacting units within a cell, a tissue or parts of an organism.Gosak et al [1] review the most recent advances in the application of multilayer networks for modeling complex biological systems, from molecular interactions within a cell to neuronal connectivity of the human brain.

show abstract

“…It is even used together with other terms to describe different levels of recreation. The Vienna Principles' definition (Kraker et al, 2016) focuses on traceability, others treat "reproducibility" and "replicability" either as interchangeable (Gentleman and Lang, 2007) or completely different terms (Goodman et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

Enabling reproducible obia with open-source software in docker containers

Knoth

Nüst

2016

GEOBIA 2016: Solutions and Synergies

View full text Add to dashboard Cite

ABSTRACT:While most approaches in Object-Based Image Analysis (OBIA) currently rely on proprietary software, the interest in free and opensource software (FOSS) for OBIA is growing. This interest stems not only from cost savings, but also from benefits concerning reproducibility and collaboration. However, technical challenges hamper practical reproducibility, especially when multiple software packages are involved. In this study, we use the Docker technology to containerise an OBIA workflow in a well-defined FOSS environment. Running the analysis inside a container eliminates the need to recreate the original software environment on the executing computer. We explore the approach using two software stacks (InterIMAGE, and QGIS in combination with Orfeo ToolBox, SAGA and Python libraries) to perform an exemplary analysis detecting destruction of buildings in bi-temporal images of a conflict area. The analysis combines feature extraction techniques with segmentation and object-based analysis to detect changes and to distinguish disappeared buildings from similarly changed non-target structures. The resulting workflow is published as FOSS comprising both the model and a ready to use Docker image including all required software and data. The presented solution advances OBIA in the following aspects: higher transparency of methodology; easier reuse and adaption of workflows; better transferability between operating systems; complete description of software environment; and easy adoption of OBIA workflows by image analysis experts and non-experts.

show abstract

What does research reproducibility mean?

Abstract: The language and conceptual framework of “research reproducibility” are nonstandard and unsettled across the sciences.

Cited by 921 publications

References 38 publications

Effects of tDCS on motor learning and memory formation: A consensus and critical position paper

Effects of tDCS on motor learning and memory formation: A consensus and critical position paper

Multilayer network modeling of integrated biological systems

Enabling reproducible obia with open-source software in docker containers

Contact Info

Product

Resources

About