Reproducibility and Replication of Experimental Particle Physics Results

Junk, T. R.

doi:10.1162/99608f92.250f995b

Cited by 10 publications

(7 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Those observations of discriminatory ability then provide the "control" strength of preference, for comparison with treatment results using the same choice under raised CO 2 (lowered pH) conditions throughout the test tank. Several versions of such experimental conditions and treatments have been developed, with differences between protocols known to affect the strength of the response change (Jutfelt et al, 2017).…”

Section: The Challenge Of Contradictory Resultsmentioning

confidence: 99%

Ideas and perspectives: When ocean acidification experiments are not the same, repeatability is not tested

et al. 2021

View full text Add to dashboard Cite

Abstract. Can experimental studies on the behavioural impacts of ocean acidification be trusted? That question was raised in early 2020 when a high-profile paper failed to corroborate previously observed responses of coral reef fish to high CO2. New information on the methodologies used in the “replicated” studies now provides a plausible explanation: the experimental conditions were substantially different. High sensitivity to test conditions is characteristic of ocean acidification research; such response variability shows that effects are complex, interacting with many other factors. Open-minded assessment of all research results, both negative and positive, remains the best way to develop process-based understanding. As in other fields, replication studies in ocean acidification are most likely to contribute to scientific advancement when carried out in a spirit of collaboration rather than confrontation.

show abstract

Section: The Challenge Of Contradictory Resultsmentioning

confidence: 99%

Ideas and perspectives: When ocean acidification experiments are not the same, repeatability is not tested

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The alternative hypothesis H 1 is that of the SM background plus a Higgs boson with a Gaussian signal, with a known width but an unknown mass m h and an unknown positive signal strength µ ≥ 0. 5 The null hypothesis lies at the µ = 0 boundary. The data x consists of the Poisson-distributed observed counts in the 30 bins shown in Figure 4 of [6].…”

Section: B Resonance Searchmentioning

confidence: 97%

“…For decades, p-values have played a vital role in the discovery of new phenomena in high-energy physics [HEP, [1][2][3][4][5] as well as in many other fields and disciplines. A p-value is the probability under a null hypothesis of observing results that are at least as "extreme" as the observed data.…”

Section: Introductionmentioning

confidence: 99%

Nested sampling for frequentist computation: fast estimation of small $p$-values

Fowlie,

Hoof,

Handley

2021

Preprint

View full text Add to dashboard Cite

We propose a novel method for computing p-values based on nested sampling (NS) applied to the sampling space rather than the parameter space of the problem, in contrast to its usage in Bayesian computation. The computational cost of NS scales as log 2 1/p, which compares favorably to the 1/p scaling for Monte Carlo (MC) simulations. For significances greater than about 4σ in both a toy problem and a simplified resonance search, we show that NS requires orders of magnitude fewer simulations than ordinary MC estimates. This is particularly relevant for high-energy physics, which adopts a 5σ gold standard for discovery. We conclude with remarks on new connections between Bayesian and frequentist computation and possibilities for tuning NS implementations for still better performance in this setting.

show abstract

“…Yet there is one additional explanation that is often overlooked in such discussions— insufficient resource investment into scientific research (Cristea & Naudet, 2019; Forscher et al, 2020). In other scientific fields such as physics, it has long been the norm to replicate and reproduce scientific results, often with teams of over 3,000 authors approving a study for publication (Junk & Lyons, 2020), but large-scale collaboration of this kind has been rare in the social sciences and is still notably absent in addiction science. This has led some researchers to call for a “CERN” for the social sciences, comprising a distributed network of hundreds of individual laboratories who work in teams to answer global questions (Chartier et al, 2018).…”

Section: Researcher Consortiamentioning

confidence: 99%

Beyond online participant crowdsourcing: The benefits and opportunities of big team addiction science.

Pennington¹,

Jones²,

Tzavella³

et al. 2022

Experimental and Clinical Psychopharmacology

View full text Add to dashboard Cite

Participant crowdsourcing platforms (e.g., MTurk, Prolific) offer numerous advantages to addiction science, permitting access to hard-to-reach populations and enhancing the feasibility of complex experimental, longitudinal, and intervention studies. Yet these are met with equal concerns about participant nonnaivety, motivation, and careless responding, which if not considered can greatly compromise data quality. In this article, we discuss an alternative crowdsourcing avenue that overcomes these issues whilst presenting its own unique advantages-crowdsourcing researchers through big team science. First, we review several contemporary efforts within psychology (e.g., ManyLabs, Psychological Science Accelerator) and the benefits these would yield if they were more widely implemented in addiction science. We then outline our own consortium-based approach to empirical dissertations: a grassroots initiative that trains students in reproducible big team addiction science. In doing so, we discuss potential challenges and their remedies, as well as providing resources to help addiction researchers develop these initiatives. Through researcher crowdsourcing, together we can answer fundamental scientific questions about substance use and addiction, build a literature that is representative of a diverse population of researchers and participants, and ultimately achieve our goal of promoting better global health. Public Health SignificanceThis special issue on "crowdsourcing methods in addiction science" focuses on best practices and emerging research that uses online participant recruitment platforms. An alternative method is that of crowdsourcing researchers through big team science. In this article, we: (a) review contemporary researcher crowdsourcing efforts and the benefits these would bring to addiction science; (b) outline our approach to teaching students in reproducible big team addiction science; and (c) evaluate challenges and their remedies, as well as providing resources, to help others develop these initiatives.

show abstract

Reproducibility and Replication of Experimental Particle Physics Results

Cited by 10 publications

References 68 publications

Ideas and perspectives: When ocean acidification experiments are not the same, repeatability is not tested

Ideas and perspectives: When ocean acidification experiments are not the same, repeatability is not tested

Nested sampling for frequentist computation: fast estimation of small $p$-values

Beyond online participant crowdsourcing: The benefits and opportunities of big team addiction science.

Contact Info

Product

Resources

About