Effect of Initial Conditions on Reproducibility of Scientific Research

Djulbegoviĉ, Benjamin; Hozo, Iztok

doi:10.5455/aim.2014.22.156-159

Cited by 7 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(In a binary logistic regression predicting replication success, we entered contextual sensitivity, subdiscipline, and their interaction as predictors. Simple effects analysis via dummy coding demonstrated that the effect of contextual sensitivity was nearly identical in magnitude for social psychology studies [odds ratio (OR) = 0.823] and cognitive psychology studies [OR = 0.892]) In other words, contextual sensitivity appears to play an important role in replication success across multiple areas of psychology, and there is good reason to believe the effect of contextual sensitivity applies to other scientific fields as well (80,81).…”

Section: Resultsmentioning

confidence: 99%

Contextual sensitivity in scientific reproducibility

Bavel

Mende-Siedlecki

Brady

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

357

313

View full text Add to dashboard Cite

In recent years, scientists have paid increasing attention to reproducibility. For example, the Reproducibility Project, a large-scale replication attempt of 100 studies published in top psychology journals found that only 39% could be unambiguously reproduced. There is a growing consensus among scientists that the lack of reproducibility in psychology and other fields stems from various methodological factors, including low statistical power, researcher's degrees of freedom, and an emphasis on publishing surprising positive results. However, there is a contentious debate about the extent to which failures to reproduce certain results might also reflect contextual differences (often termed "hidden moderators") between the original research and the replication attempt. Although psychologists have found extensive evidence that contextual factors alter behavior, some have argued that context is unlikely to influence the results of direct replications precisely because these studies use the same methods as those used in the original research. To help resolve this debate, we recoded the 100 original studies from the Reproducibility Project on the extent to which the research topic of each study was contextually sensitive. Results suggested that the contextual sensitivity of the research topic was associated with replication success, even after statistically adjusting for several methodological characteristics (e.g., statistical power, effect size). The association between contextual sensitivity and replication success did not differ across psychological subdisciplines. These results suggest that researchers, replicators, and consumers should be mindful of contextual factors that might influence a psychological process. We offer several guidelines for dealing with contextual sensitivity in reproducibility.replication | reproducibility | context | psychology | meta-science I n recent years, scientists have paid increasing attention to reproducibility. Unsuccessful attempts to replicate findings in genetics (1), pharmacology (2), oncology (3), biology (4), and economics (5) have given credence to previous speculation that most published research findings are false (6). Indeed, since the launch of the clinicaltrials.gov registry in 2000, which forced researchers to preregister their methods and outcome measures, the percentage of large heart-disease clinical trials reporting significant positive results plummeted from 57% to a mere 8% (7). The costs of such irreproducible preclinical research, estimated at $28 billion in the United States (8), are staggering. In a similar vein, psychologists have expressed growing concern regarding the reproducibility and validity of psychological research (e.g., refs. 9-14). This emphasis on reproducibility has produced a number of failures to replicate prominent studies, leading professional societies and government funding agencies such as the National Science Foundation to form subcommittees promoting more robust research practices (15).The Reproducibility Project in psychology has b...

show abstract

Section: Resultsmentioning

confidence: 99%

Contextual sensitivity in scientific reproducibility

Bavel

Mende-Siedlecki

Brady

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

357

313

View full text Add to dashboard Cite

show abstract

“…In-depth analysis of the robustness of these relations is therefore contributing to the quality of information provided by them. A similar proposal, in the context of reproducible research, is presented in Djulbegovic and Hozo (2014). Overall, this work is about increasing the quality of information provided by a BN.…”

Section: Discussionmentioning

confidence: 94%

Bayesian Networks in Survey Data: Robustness and Sensitivity Issues

Cugnata

Kenett

Salini

2016

Journal of Quality Technology

View full text Add to dashboard Cite

Bayesian networks (BN) implement a graphical model structure known as a directed acyclic graph (DAG) that is popular in statistics, machine learning, and artificial intelligence. They enable an e↵ective representation and computation of a joint probability distribution (JPD) over a set of random variables. The paper focuses on the selection of a robust network structure according to di↵erent learning algorithms and the measure of arc strength using resampling techniques. Moreover, it shows how 'what-if' sensitivity scenarios are generated with BN using hard and soft evidence in the framework of predictive inference. Establishing a robust network structure and using it for decision support are two essential enablers for ecient and e↵ective applications of BN to improvements of products and processes. A customer-satisfaction survey example is presented and R scripts are provided.

show abstract

“…16 Of particular importance is the ability to define unambiguously the initial conditions prevailing when the process was performed. 17 Perhaps most critically is the way in which research is actually disseminated and reproduced.…”

Section: The Current State Of Automationmentioning

confidence: 99%

Highly Versatile Cloud-Based Automation Solution for the Remote Design and Execution of Experiment Protocols during the COVID-19 Pandemic

Zucchelli¹,

Horak²,

Skinner³

2021

SLAS Technology

View full text Add to dashboard Cite

There is an urgent need to accelerate the development and validation of both diagnostics and vaccines for COVID-19. These priorities are challenging both public and private sector research groups around the world and have shone a spotlight on both existing bottlenecks in the research workflows involved as well as on the implications of having to do much of this work remotely because of enforced social distancing and lockdown measures. The ability to respond quickly to rapidly evolving events, coupled with an emerging understanding of the disease and its pathology, as well as different mutations of the virus, necessitates a highly flexible liquid-handling automation solution that is amenable to rapid switching between different assay workflows and processes to be exploited tactically as needed. In addition, the use of cloud-based software imparts a unique benefit in enabling multiple research groups and remote technical staff around the world to have ready access to the same protocols in real-time without delays, down to the required level of detail, sharing methods and data (for example, in faster clinical trials). Informed by a recent use case, this article explores these issues alongside the recent development and deployment of an automation solution, whose unique approach in terms of both its cloud-native software and its highly modular hardware aligns especially well with achieving the challenge set by this new frontier in the bioanalytical laboratory.

show abstract

Effect of Initial Conditions on Reproducibility of Scientific Research

Cited by 7 publications

References 13 publications

Contextual sensitivity in scientific reproducibility

Contextual sensitivity in scientific reproducibility

Bayesian Networks in Survey Data: Robustness and Sensitivity Issues

Highly Versatile Cloud-Based Automation Solution for the Remote Design and Execution of Experiment Protocols during the COVID-19 Pandemic

Contact Info

Product

Resources

About