Potential effects of lowering the threshold of statistical significance in the field of chronic rhinosinusitis – A meta-research on published randomized controlled trials over last decade

Thakur, Pooja; Jha, Vivek

doi:10.1016/j.bjorl.2021.11.004

Cited by 2 publications

(2 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In accordance with the rst MR assumption, a strict threshold should be employed in order to identify variants with a robust association with the exposure. However, more lenient threshold values were included here to allow for the comparison of both the number and strength of included instrumental variables identi ed for each ancestry group (39,40). When a data source has a small sample size or has limited available instruments, as is seen more commonly in non-European datasets, researchers may consider employing less stringent alpha thresholds to allow for more variants to be included in the analysis.…”

Section: Genetic Instrumentsmentioning

confidence: 99%

Genetic Insights into Coronary Artery Disease in Underrepresented Populations: Assessing Two-Sample Mendelian Randomization across Diverse Ancestry Populations

Fatumo,

Silva,

Hughes

et al. 2024

Preprint

View full text Add to dashboard Cite

Understanding the causal effect of modifiable risk factors on a disease is crucial for aiding and shaping public health policies, identifying targets of interventions and advancing our global understanding of health and diseases. With this however, the disproportionate representation of non-European ancestries in research has raised important questions regarding the transferability and reliability of genetic findings on a diverse global scale. In this study, we investigated the feasibility of conducting a two-sample Mendelian randomization (MR) analysis in populations of diverse ancestries, focusing on both methodological challenges and biologically differences when data from European, East Asian, South Asian and African ancestry populations were compared against each other. Employing data generated from large-scale genome-wise association studies (GWAS), we chose to compare the causal effects of lipid traits, blood pressure, body-mass index, type-2 diabetes and kidney function on coronary artery disease. Due to an insufficient number of identified single-nucleotide polymorphisms in non-European data when strict alpha thresholds were employed, we were not able to conduct MR analyses across all ancestry populations until the threshold was relaxed. We found that allowing for a lenient inclusion threshold and extending the MR methodology to be more inclusive of non-European data resulted in an increase in weak instrument bias, resulting in imprecise estimates and a reduced ability to detect true causal effects. Notably, our results showed causal associations known to be inconsistent with established findings, specifically between lipid traits in South Asian populations compared to European ancestry populations. These findings reiterate the urgent need for independent large-scale GWAS in non-European populations, to improve the power and reliability of MR studies as well as to develop methods which take into account population-specific effects.

show abstract

Section: Genetic Instrumentsmentioning

confidence: 99%

Genetic Insights into Coronary Artery Disease in Underrepresented Populations: Assessing Two-Sample Mendelian Randomization across Diverse Ancestry Populations

Fatumo,

Silva,

Hughes

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…In a series of studies, Vassar and colleagues examined the impact of changing the threshold on randomized controlled trials (RCTs) published in general medical, orthopaedic trauma, and orthopaedic sports medicine journals [24][25][26]. The results of these studies are summarized in Table 1, along with a study by Thakur and Jha [27] that examined changing the P value threshold on results from 123 RCTs pertaining to chronic rhinosinusitis and a study by Khan et al [28] that focused on 72 RCTS from high impact general medical and cardiology journals. Across the five studies, the range of P values that retained statistical significance with a 0.005 threshold was 38.9-70.7%.…”

Section: Introductionmentioning

confidence: 99%

Impact of redefining statistical significance on P-hacking and false positive rates: An agent-based model

Fitzpatrick,

Gorman,

Trombatore

2024

PLoS ONE

View full text Add to dashboard Cite

In recent years, concern has grown about the inappropriate application and interpretation of P values, especially the use of P<0.05 to denote “statistical significance” and the practice of P-hacking to produce results below this threshold and selectively reporting these in publications. Such behavior is said to be a major contributor to the large number of false and non-reproducible discoveries found in academic journals. In response, it has been proposed that the threshold for statistical significance be changed from 0.05 to 0.005. The aim of the current study was to use an evolutionary agent-based model comprised of researchers who test hypotheses and strive to increase their publication rates in order to explore the impact of a 0.005 P value threshold on P-hacking and published false positive rates. Three scenarios were examined, one in which researchers tested a single hypothesis, one in which they tested multiple hypotheses using a P<0.05 threshold, and one in which they tested multiple hypotheses using a P<0.005 threshold. Effects sizes were varied across models and output assessed in terms of researcher effort, number of hypotheses tested and number of publications, and the published false positive rate. The results supported the view that a more stringent P value threshold can serve to reduce the rate of published false positive results. Researchers still engaged in P-hacking with the new threshold, but the effort they expended increased substantially and their overall productivity was reduced, resulting in a decline in the published false positive rate. Compared to other proposed interventions to improve the academic publishing system, changing the P value threshold has the advantage of being relatively easy to implement and could be monitored and enforced with minimal effort by journal editors and peer reviewers.

show abstract

Potential effects of lowering the threshold of statistical significance in the field of chronic rhinosinusitis – A meta-research on published randomized controlled trials over last decade

Cited by 2 publications

References 36 publications

Genetic Insights into Coronary Artery Disease in Underrepresented Populations: Assessing Two-Sample Mendelian Randomization across Diverse Ancestry Populations

Genetic Insights into Coronary Artery Disease in Underrepresented Populations: Assessing Two-Sample Mendelian Randomization across Diverse Ancestry Populations

Impact of redefining statistical significance on P-hacking and false positive rates: An agent-based model

Contact Info

Product

Resources

About