Nonindependence and sensitivity analyses in ecological and evolutionary meta‐analyses

Abstract

Meta-analysis is an important tool for synthesizing research on a variety of topics in ecology and evolution, including molecular ecology, but can be susceptible to nonindependence. Nonindependence can affect two major interrelated components of a meta-analysis: (i) the calculation of effect size statistics and (ii) the estimation of overall meta-analytic estimates and their uncertainty. While some solutions to nonindependence exist at the statistical analysis stages, there is little advice on what to do when complex analyses are not possible, or when studies with nonindependent experimental designs exist in the data. Here we argue that exploring the effects of procedural decisions in a meta-analysis (e.g. inclusion of different quality data, choice of effect size) and statistical assumptions (e.g. assuming no phylogenetic covariance) using sensitivity analyses are extremely important in assessing the impact of nonindependence. Sensitivity analyses can provide greater confidence in results and highlight important limitations of empirical work (e.g. impact of study design on overall effects). Despite their importance, sensitivity analyses are seldom applied to problems of nonindependence. To encourage better practice for dealing with nonindependence in meta-analytic studies, we present accessible examples demonstrating the impact that ignoring nonindependence can have on meta-analytic estimates. We also provide pragmatic solutions for dealing with nonindependent study designs, and for analysing dependent effect sizes. Additionally, we offer reporting guidelines that will facilitate disclosure of the sources of nonindependence in meta-analyses, leading to greater transparency and more robust conclusions.

Multiple drivers of contrasting diversity–invasibility relationships at fine spatial grains

“…, Noble et al. ). However, one limitation of a random‐effects model in meta‐analysis is that it assumes that each study contributes only one effect size to the data set, which is not the case for most meta‐analyses in ecology and evolution (Nakagawa and Santos , Noble et al.…”

“…If multiple performance measures were provided for the same individual (or colony), we randomly selected one measure to avoid issues arising from non‐independence (Gurevitch and Hedges , Noble et al. ). Likewise, if more than one invasibility metric was presented for focal taxa within the same plot (e.g., a performance metric for a single invasive species and total invasive species richness), we randomly selected one measure of invasibility.…”

“…However, one limitation of a random‐effects model in meta‐analysis is that it assumes that each study contributes only one effect size to the data set, which is not the case for most meta‐analyses in ecology and evolution (Nakagawa and Santos , Noble et al. ), or here. Omitting multiple effect sizes from a study reduces statistical power and could result in a loss of information (Nakagawa and Santos ).…”

“…, Noble et al. ), to assess how sensitive our estimates of overall effect sizes based on random‐effects models were to inclusion of multiple effect sizes from a single study, that is, inclusion of non‐independent effect sizes (Appendix : Fig. S3; see Assessing publication bias and other sensitivity analyses for details).…”

“…In other words, we used “multilevel meta‐regressions” (Nakagawa and Santos , Noble et al. ). However, to examine the effect of invasibility metric on the paradox at fine grains, we also conducted a chi‐squared (χ 2 ) contingency test to assess whether there was bias in how invasibility was quantified (i.e., using performance metrics or species richness) between study types.…”