Misinterpretation of P Values and Statistical Power Creates a False Sense of Certainty: Statistical Significance, Lack of Significance, and the Uncertainty Challenge

Cote, Mark P.; Lubowitz, James H.; Brand, Jefferson C.; Rossi, Michael J.

doi:10.1016/j.arthro.2021.02.010

Cited by 17 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The purpose is to express differences in an intuitive manner, less susceptible to dichotomizing results as significant or not significant. 17 For reference, a P value of .05 translates to an S value of 4.32. This correlates with the chances of a flipped coin landing on the same side between 4 and 5 consecutive times.…”

Section: Methodsmentioning

confidence: 99%

Pathologic Findings on Hip Arthroscopy in High-Level Athletes Competing in Flexibility Sports

et al. 2022

View full text Add to dashboard Cite

Background: Athletes who compete in flexibility sports (FS) place unique demands on their hip joints because of the supraphysiologic range of motion required. Purpose: To compare the pathologic features, outcomes, and return-to-sports (RTS) rates of high-level athletes participating in FS who underwent hip arthroscopy for femoroacetabular impingement syndrome (FAIS) and labral tear against a propensity score–matched cohort of high-level athletes participating in non–flexibility sports (NFS). Study Design: Cohort study; Level of evidence, 3. Methods: Data were prospectively collected and retrospectively reviewed for high-level athletes who underwent primary hip arthroscopy for FAIS from April 2008 to December 2018. Patients who participated in FS such as dancing, gymnastics, martial arts, figure skating, and cheerleading were propensity score matched by body mass index, age at time of surgery, sex, sports competition level, and labral treatment to a cohort of high-level athletes participating in all other sports, such as distance running, soccer, volleyball, and softball. Baseline patient characteristics, intraoperative findings, and surgical procedures were compared. Minimum 2-year patient-reported outcome measures were compared for the modified Harris Hip Score, Nonarthritic Hip Score, Hip Outcome Score–Sport Specific Subscale, and visual analog scale for pain and satisfaction. Rates of secondary surgery and RTS were compared. Results: A total of 47 patients (50 hips) who participated in FS were included and propensity score matched to 130 patients (150 hips) who participated in NFS. Follow-up time was 37.5 ± 10.4 months (mean ± SD). Most patients (96.0%) were female with a mean age of 19.5 ± 7.3 years. FS athletes had significantly higher rates of femoral head cartilage lesions (Outerbridge ≥2; 12.0% vs 2.0%; P = .008) and ligamentum teres tears (48% vs 26%; P = .003). FS and NFS athletes demonstrated significant clinical improvements after surgery for all patient-reported outcome measures. Of the patients who attempted, 34 (75.6%) participating in FS were able to RTS while 11 (24.4%) were not because of ongoing hip issues. This was not significantly different than the NFS group ( P = .073). Conclusion: High-level athletes who participated in FS and were treated for FAIS with hip arthroscopy exhibited higher rates of femoral head cartilage lesions and ligamentum teres tears requiring debridement when compared with a benchmark group of athletes who participated in other sports. Despite this, both groups demonstrated similar improvements in outcome scores and comparable rates of RTS at minimum 2-year follow-up.

show abstract

Section: Methodsmentioning

confidence: 99%

Pathologic Findings on Hip Arthroscopy in High-Level Athletes Competing in Flexibility Sports

et al. 2022

View full text Add to dashboard Cite

show abstract

“…GTPBP1 has been shown to play a role in translation and maintaining neuronal homeostasis which may represent a potential gene of interest not identified as significant due to sample size. [80][81][82] FBX015 encodes a member of the F-box family of proteins which mediates protein-protein interactions through facilitating ubiquitination of proteins. 83 LINC02582 corresponds to a gene that produces a nonprotein coding RNA, and such RNAs can play a key role in gene regulation.…”

Section: Discussionmentioning

confidence: 99%

“…The strongest association we identified was with rs67256809 in the intergenic region between LINC02582 and FBXO15 , while the strongest association within a gene footprint was for the indel variant rs577081672 in the GTPBP1 gene. GTPBP1 has been shown to play a role in translation and maintaining neuronal homeostasis which may represent a potential gene of interest not identified as significant due to sample size 80–82 . FBX015 encodes a member of the F-box family of proteins which mediates protein-protein interactions through facilitating ubiquitination of proteins 83 .…”

Section: Discussionmentioning

confidence: 99%

Evaluation of Genetic and Non-Genetic Risk Factors for Degenerative Cervical Myelopathy

Shlykov¹,

Giles²,

Kelly³

et al. 2023

Spine

View full text Add to dashboard Cite

Study Design. Cohort study. Objective. We aimed to evaluate the associations of genetic and nongenetic factors with degenerative cervical myelopathy (DCM). Summary of Background Data. There is mounting evidence for an inherited predisposition for DCM, but uncertainty remains regarding specific genetic markers involved. Similarly, nongenetic factors are thought to play a role. Materials and Methods. Using diagnosis codes from hospital records linked to the UK Biobank cohort, patients with cervical spondylosis were identified followed by the identification of a subset with DCM. Nongenetic variables evaluated included age, sex, race, Townsend deprivation index, body mass index, occupational demands, osteoporosis, and smoking. Genome-wide association analyses were conducted using logistic regression adjusted for age, sex, population principal components, and follow-up. Results. A total of 851 DCM cases out of 2787 cervical spondylosis patients were identified. Several nongenetic factors were independently associated with DCM including age [odds ratio (OR)=1.11, 95% CI=1.01–1.21, P=0.024], male sex (OR=1.63, 95% CI=1.37–1.93, P<0.001), and relative socioeconomic deprivation (OR=1.03, 95% CI=1.00–1.06, P=0.030). Asian race was associated with lower DCM risk (OR=0.44, 95% CI=0.22–0.85, P=0.014). We did not identify genome-wide significant (≤5×10−8) single-nucleotide polymorphisms (SNPs) associated with DCM. The strongest genome-wide signals were at SNP rs67256809 in the intergenic region of the genes LINC02582 and FBXO15 on chromosome 18 (P=1.12×10−7) and rs577081672 in the GTPBP1 gene on chromosome 22 (P=2.9×10−7). No SNPs reported in prior DCM studies were significant after adjusting for replication attempts. Conclusions. Increasing age, male sex, and relative socioeconomic deprivation were identified as independent risk factors for DCM, whereas Asian race was inversely associated. SNPs of potential interest were identified in GTPBP1 and an intergenic region on chromosome 18, but these associations did not reach genome-wide significance. Identification of genetic and nongenetic DCM susceptibility markers may guide understanding of DCM disease processes, inform risk, guide prevention and potentially inform surgical outcomes. Level of Evidence. Prognostic level III.

show abstract

“…Some authors suggest that the p-value inappropriately dichotomizes data as significant or insignificant, creating a false sense of confidence in data that reach the p-value threshold. These authors suggest adopting the use of an alternative “surprise” or “s-value” that converts a p-value into the number of successive identical results of flips of a fair coin [ 11 ]. However, no clear alternative has been adopted as a standard of practice at this time.…”

Section: Discussionmentioning

confidence: 99%

Misinterpretations of Significance Testing Results Near the P-Value Threshold in the Urologic Literature

Manda¹,

Kuchakulla²,

Hochu³

et al. 2023

Cureus

View full text Add to dashboard Cite

Background The outcome of a statistical test is to accept or reject a null hypothesis. Reporting a metric as “trending toward significance” is a misinterpretation of the p-value. Studies highlighting the prevalence of statistical errors in the urologic literature remain scarce. We evaluated abstracts from 15 urology journals published within the years 2000-2021 and provided a quantitative measure of a common statistical mistake-misconstruing the function of null hypothesis testing by reporting “a trend toward significance.” Materials and methods We performed an audit of 15 urology journals, looking at articles published from January 1, 2000, to January 1, 2022. A word recognition function in Microsoft Excel was utilized to identify the use of the word “trend” in the abstracts. Each use of the word “trend” was manually investigated by two authors to determine whether it was improperly used in describing non-statistically significant data as trending toward significance. Statistics and data analysis were performed using Python libraries: pandas, scipy.stats, and seaborn. Results This study included 101,134 abstracts from 15 urology journals. Within those abstracts, the word “trend” was used 2,509 times, 572 uses of which were describing non-statistically significant data as trending toward significance. There was a statistically significant difference in the rate of errors between the 15 journals (p < 0.01). The highest rate of improper use of the word “trend” was found in Bladder Cancer with a rate of 1.6% (p < 0.01) of articles. The lowest rate of improper use was found in European Urology, with a rate of 0.3% (p < 0.01). Our analysis found a moderate correlation between the number of articles published and the number of misuses of the word "trend" within each journal and across all journals every year (r = 0.61 and 0.70, respectively). Conclusion The overall rate of p-value misinterpretation never exceeded 2% of articles in each journal. There is significance in the difference in misinterpretation rates between the different journals. Authors' utilization of the word “trend” describing non-significant p-values as being near significant should be used with caution.

show abstract

Misinterpretation of P Values and Statistical Power Creates a False Sense of Certainty: Statistical Significance, Lack of Significance, and the Uncertainty Challenge

Cited by 17 publications

References 17 publications

Pathologic Findings on Hip Arthroscopy in High-Level Athletes Competing in Flexibility Sports

Pathologic Findings on Hip Arthroscopy in High-Level Athletes Competing in Flexibility Sports

Evaluation of Genetic and Non-Genetic Risk Factors for Degenerative Cervical Myelopathy

Misinterpretations of Significance Testing Results Near the P-Value Threshold in the Urologic Literature

Contact Info

Product

Resources

About