A Guide for Calculating Study-Level Statistical Power for Meta-Analyses

Quintana, Daniel

doi:10.1177/25152459221147260

Cited by 16 publications

(13 citation statements)

References 72 publications

(122 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…All analyses were performed in R (Version 4.2.1; R Core Team, 2022) using the following packages: metafor (Version 3.8 − 1; Viechtbauer, 2010), dmetar (Version 0.0.9; Harrer et al, 2019), meta (Version 6.0 − 0; Schwarzer, 2007), metaforest (Version 0.1.3; van Lissa, 2020), metaviz (Version 0.3.1; Kossmeier et al, 2020), metameta (Version 0.2; Quintana, 2023), orchaRd (Version 2.0; Nakagawa et al, 2021), PublicationBias (Version 2.2.0; Mathur & VanderWeele, 2020), TOSTER (Version 0.8.0; Lakens, 2017), weightr (Version 2.0.2; Coburn et al, 2019), clubSandwich (Version 0.5.8; Pustejovsky & Tipton, 2022), and dplyr (Version 1.1.1; Wickham et al, 2015).…”

Section: Methodsmentioning

confidence: 99%

Committed (dis)honesty: A systematic meta-analytic review of the divergent effects of social commitment to individuals or honesty oaths on dishonest behavior.

Zickfeld,

Karg,

Engen

et al. 2024

Psychological Bulletin

View full text Add to dashboard Cite

People feel committed to other individuals, groups, organizations, or moral norms in many contexts of everyday life. Such social commitment can lead to positive outcomes, such as increased job satisfaction or relationship longevity; yet, there can also be detrimental effects to feeling committed. Recent high-profile cases of fraud or corruption in companies like Enron or Volkswagen are likely influenced by strong commitment to the organization or coworkers. Although social commitment might increase dishonest behavior, there is little systematic knowledge about when and how this may occur. In the present project, we reviewed 20,988 articles, focusing on studies that experimentally manipulated social commitment and measured dishonest behavior. We retained 445 effect sizes from 121 articles featuring a total of 91,683 participants across 33 countries. We found no evidence that social commitment increases or reduces dishonest behavior in general. Nonetheless, we did find evidence that the effect strongly depends on the target of the commitment. Feeling committed to other individuals or groups reduces honest behavior (g = −0.17 [−0.24, −0.11]), whereas feeling committed to honesty norms through honesty oaths or pledges increases honest behavior (g = 0.27 [0.19, 0.36]). The analysis identified several moderating variables and detected some degree of publication bias across effects. Our findings highlight the diverging effects of different forms of social commitment on dishonest behavior and suggest a combination of the different forms of commitment could be a possible means to combat corruption and dishonest behavior in the organizational context. Public Significance StatementThis meta-analysis reveals that social commitment can impact dishonest behavior in various ways. Feeling committed or interacting with other individuals showed an increase in dishonesty, whereas committing to an honesty norm (by signing an honesty oath, for example) showed an increase in honesty. These effects were small and subject to different situational moderations. We propose that a combination of both types of commitment, commitment to loyalty and commitment to honesty norms, might be important in achieving good teamwork while at the same time reducing dishonest behavior.

show abstract

Section: Methodsmentioning

confidence: 99%

Committed (dis)honesty: A systematic meta-analytic review of the divergent effects of social commitment to individuals or honesty oaths on dishonest behavior.

Zickfeld,

Karg,

Engen

et al. 2024

Psychological Bulletin

View full text Add to dashboard Cite

show abstract

“…This method converts significant and nonsignificant p -values reported in a literature into two-tailed z -scores, and uses the distribution of z -scores to calculate two estimates of average power using finite mixture modeling: the expected discovery rate, which is the percentage of studies predicted to be significant based on the average power of published studies and the expected replication rate, which is the average power of the studies entered, which is also an estimate of the percent of the studies that one would expect to replicate if one performed the studies in exactly the same way as they were done before. Second, we used a range of hypothetical effect sizes to estimate statistical power using the R package metameta [ 58 ]. This package allows researchers to estimate the statistical power of the studies included in the meta-analysis by using (a) a range of hypothetical effect sizes and (b) the meta-analytic effect size estimate as the true effect size.…”

Section: Methodsmentioning

confidence: 99%

Assessing the Evidential Value of Mental Fatigue and Exercise Research

Holgado,

Mesquida,

Román-Caballero

2023

Sports Med

View full text Add to dashboard Cite

It has often been reported that mental exertion, presumably leading to mental fatigue, can negatively affect exercise performance; however, recent findings have questioned the strength of the effect. To further complicate this issue, an overlooked problem might be the presence of publication bias in studies using underpowered designs, which is known to inflate false positive report probability and effect size estimates. Altogether, the presence of bias is likely to reduce the evidential value of the published literature on this topic, although it is unknown to what extent. The purpose of the current work was to assess the evidential value of studies published to date on the effect of mental exertion on exercise performance by assessing the presence of publication bias and the observed statistical power achieved by these studies. A traditional meta-analysis revealed a Cohen’s dz effect size of − 0.54, 95% CI [− 0.68, − 0.40], p < .001. However, when we applied methods for estimating and correcting for publication bias (based on funnel plot asymmetry and observed p-values), we found that the bias-corrected effect size became negligible with most of publication-bias methods and decreased to − 0.36 in the more optimistic of all the scenarios. A robust Bayesian meta-analysis found strong evidence in favor of publication bias, BFpb > 1000, and inconclusive evidence in favor of the effect, adjusted dz = 0.01, 95% CrI [− 0.46, 0.37], BF10 = 0.90. Furthermore, the median observed statistical power assuming the unadjusted meta-analytic effect size (i.e., − 0.54) as the true effect size was 39% (min = 19%, max = 96%), indicating that, on average, these studies only had a 39% chance of observing a significant result if the true effect was Cohen’s dz = − 0.54. If the more optimistic adjusted effect size (− 0.36) was assumed as the true effect, the median statistical power was just 20%. We conclude that the current literature is a useful case study for illustrating the dangers of conducting underpowered studies to detect the effect size of interest.

show abstract

“…To quantify the study-level statistical power across a range of true effect sizes, a ʻFirepowerʼ plot was generated using functions from the ‘metameta’ package (Supplementary Fig. 2, Supplemental Digital Content 5, http://links.lww.com/JS9/B196 ) 22 . This analysis provided valuable insights into the statistical robustness of the included meta-analyses 22 .…”

Section: Methodsmentioning

confidence: 99%

“…2, Supplemental Digital Content 5, http://links.lww.com/JS9/B196 ) 22 . This analysis provided valuable insights into the statistical robustness of the included meta-analyses 22 .…”

Section: Methodsmentioning

confidence: 99%

Impact of antibiotic-coated sutures on surgical site infections: a second-order meta-analysis

Suleiman,

Abbass,

Hossain

et al. 2023

International Journal of Surgery

View full text Add to dashboard Cite

Background: Surgical site infections (SSIs) pose a global challenge, impacting patients and healthcare expenditures. This second-order meta-analysis endeavors to assess the efficacy of antibiotic sutures in averting SSIs by amalgamating data from various meta-studies. Materials and Methods This research adhered to the PRISMA 2020 guidelines. The quality and comprehensiveness of the encompassed meta-analyses were assessed through the QUOROM checklist and AMSTAR techniques. The primary study overlap was evaluated via measures such as pairwise intersection heat maps, Corrected Covered Area (CCA), and the citation matrix of evidence. The statistical power at the study level was determined utilizing the meta-meta package. Data synthesis employed random and fixed effects models at a 95% confidence interval. A meta-regression analysis was conducted to explore potential correlations between the CDC classification of surgical site infections, trial types, and the observed effect sizes in the studies. Results This investigation revealed a significant reduction in SSI rates due to antimicrobial-coated sutures, evidenced by a relative risk (RR) of 0.68 (95% CI: 0.59 to 0.76), with a prediction interval of 0.38 to 1.19. The analysis encompassed 18 studies with 22 meta-analyses, demonstrating a median QUOROM score of 13.6 out of 18 and an AMSTAR score of 9.1 out of 11. The presence of moderate heterogeneity was noted (Q=106.611, I square=54.038%), with non-randomized controlled trials (RCTs) exhibiting an RR of 0.56 (95% CI: 0.39 to 0.80), and RCTs displaying an RR of 0.71 (95% CI: 0.63 to 0.81). Subgroup analysis unveiled variable RR reductions for specific surgical procedures. Conclusion Antimicrobial-coated sutures offer a promising approach to mitigating surgical site infection risk (SSIs). However, their efficacy is optimally realized when employed in conjunction with other robust practices.

show abstract

A Guide for Calculating Study-Level Statistical Power for Meta-Analyses

Cited by 16 publications

References 72 publications

Committed (dis)honesty: A systematic meta-analytic review of the divergent effects of social commitment to individuals or honesty oaths on dishonest behavior.

Committed (dis)honesty: A systematic meta-analytic review of the divergent effects of social commitment to individuals or honesty oaths on dishonest behavior.

Assessing the Evidential Value of Mental Fatigue and Exercise Research

Impact of antibiotic-coated sutures on surgical site infections: a second-order meta-analysis

Contact Info

Product

Resources

About