Reproducibility and Replication of Experimental Particle Physics Results

Junk, Thomas R.; Lyons, Louis

doi:10.48550/arxiv.2009.06864

Cited by 3 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…a 5% error rate, giving an error rate of 1 − 0.95 n . 17 This issue is illustrated in fig. 1 using the B-physics observable φ s , which is a well-measured phase characterising CP violation in B s meson decays.…”

Section: /10mentioning

confidence: 99%

Simple and statistically sound recommendations for analysing physical theories

AbdusSalam,

Agocs,

Allanach

et al. 2020

Preprint

View full text Add to dashboard Cite

Physical theories that depend on many parameters or are tested against data from many different experiments pose unique challenges to parameter estimation. Many models in particle physics, astrophysics and cosmology fall into one or both of these categories. These issues are often sidestepped with very simplistic and statistically unsound ad hoc methods, involving naive intersection of parameter intervals estimated by multiple experiments, and random or grid sampling of model parameters. Whilst these methods are easy to apply, they exhibit pathologies even in low-dimensional parameter spaces, and quickly become problematic to use and interpret in higher dimensions. In this article we give clear guidance for going beyond these rudimentary procedures, suggesting some simple methods for performing statistically sound inference, and recommendations of readily-available software tools and standards that can assist in doing so. Our aim is to provide physicists with recommendations for reaching correct scientific conclusions, with only a modest increase in analysis burden.

show abstract

Section: /10mentioning

confidence: 99%

Simple and statistically sound recommendations for analysing physical theories

AbdusSalam,

Agocs,

Allanach

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…This may explain, in part, why so many nominally significant medical research results are not borne out by subsequent studies. For comparison, in particle physics, a common requirement is 22 bits of refutational information (p ≤ 2.87 × 10 −7 ), which corresponds to obtaining all tails in 22 tosses of a fair coin [45].…”

Section: Confirmations and Refutationsmentioning

confidence: 99%

Interpreting Randomized Controlled Trials

Msaouel,

Lee,

Thall

2023

Cancers

View full text Add to dashboard Cite

This article describes rationales and limitations for making inferences based on data from randomized controlled trials (RCTs). We argue that obtaining a representative random sample from a patient population is impossible for a clinical trial because patients are accrued sequentially over time and thus comprise a convenience sample, subject only to protocol entry criteria. Consequently, the trial’s sample is unlikely to represent a definable patient population. We use causal diagrams to illustrate the difference between random allocation of interventions within a clinical trial sample and true simple or stratified random sampling, as executed in surveys. We argue that group-specific statistics, such as a median survival time estimate for a treatment arm in an RCT, have limited meaning as estimates of larger patient population parameters. In contrast, random allocation between interventions facilitates comparative causal inferences about between-treatment effects, such as hazard ratios or differences between probabilities of response. Comparative inferences also require the assumption of transportability from a clinical trial’s convenience sample to a targeted patient population. We focus on the consequences and limitations of randomization procedures in order to clarify the distinctions between pairs of complementary concepts of fundamental importance to data science and RCT interpretation. These include internal and external validity, generalizability and transportability, uncertainty and variability, representativeness and inclusiveness, blocking and stratification, relevance and robustness, forward and reverse causal inference, intention to treat and per protocol analyses, and potential outcomes and counterfactuals.

show abstract

Interpreting Randomized Controlled Trials

Msaouel,

Lee,

Thall

2023

Preprint

View full text Add to dashboard Cite

This article describes rationales and limitations for making inferences based on data from randomized controlled trials (RCTs). We argue that obtaining a representative random sample from a patient population is impossible for a clinical trial because patients are accrued sequentially over time and thus comprise a convenience sample, subject only to protocol entry criteria. Consequently, the trial’s sample is unlikely to represent a definable patient population. We use causal diagrams to illustrate the difference between random allocation of interventions within a clinical trial sample and true simple or stratified random sampling, as done in surveys. We argue that group-specific statistics, such as a median survival time estimate for a treatment arm in an RCT, have limited meaning as estimates of larger patient population parameters. In contrast, random allocation between interventions facilitates comparative causal inferences about between-treatment effects, such as hazard ratios or differences between probabilities of response. Comparative inferences also require the assumption of transportability from a clinical trial’s convenience sample to a targeted patient population. We focus on the consequences and limitations of randomization procedures in order to clarify the distinctions between pairs of complementary concepts of fundamental importance to data science and RCT interpretation. These include internal and external validity, generalizability and transportability, uncertainty and variability, representativeness and inclusiveness, blocking and stratification, relevance and robustness, forward and reverse causal inference, intention to treat and per protocol analyses, and potential outcomes and counterfactuals.

show abstract

Reproducibility and Replication of Experimental Particle Physics Results

Cited by 3 publications

References 18 publications

Simple and statistically sound recommendations for analysing physical theories

Simple and statistically sound recommendations for analysing physical theories

Interpreting Randomized Controlled Trials

Interpreting Randomized Controlled Trials

Contact Info

Product

Resources

About