Are experiment sample sizes adequate to detect biologically important interactions between multiple stressors?

Abstract

1. Most ecosystems are subject to co-occurring, anthropogenically driven changes and understanding how these multiple stressors interact is a pressing concern. Stressor interactions are typically studied using null models, with the additive and multiplicative null expectation being those most widely applied. Such approaches classify interactions as being synergistic, antagonistic, reversal, or indistinguishable from the null expectation. Despite their wide-spread use, there has been no thorough analysis of these null models, nor a systematic test of the robustness of their results to sample size or sampling error in the estimates of the responses to stressors. 2. We use data simulated from food web models where the true stressor interactions are known, and analytical results based on the null model equations to uncover how (i) sample size, (ii) variation in biological responses to the stressors and (iii) statistical significance, affect the ability to detect non-null interactions. 3. Our analyses lead to three main results. Firstly, it is clear the additive and multiplicative null models are not directly comparable, and over one third of all simulated interactions had classifications that were model dependent. Secondly, both null models have weak power to correctly classify interactions at commonly implemented sample sizes (i.e., ≤6 replicates), unless data uncertainty is unrealistically low. This means all but the most extreme interactions are indistinguishable from the null model expectation. Thirdly, we show that increasing sample size increases the power to detect the true interactions but only very slowly. However, the biggest gains come from increasing replicates from 3 up to 25 and we provide an R function for users to determine sample sizes required to detect a critical effect size of biological interest for the additive model. 4. Our results will aid researchers in the design of their experiments and the subsequent interpretation of results. We find no clear statistical advantage of using one null model over the other and argue null model choice should be based on biological relevance rather than statistical properties. However, there is a pressing need to increase experiment sample sizes otherwise many biologically important synergistic and antagonistic stressor interactions will continue to be missed.

multiplestressR: An R package to analyse factorial multiple stressor data using the additive and multiplicative null models

“…The multiplicative null model is becoming increasingly popular within multiple stressor research (e.g., Harvey et al 2013;Gomez Isaza et al 2020), and is recognised as potentially being more ecologically realistic than the additive null model (Schäfer & Piggott, 2018). Here, the effect_size_multiplicative function implements the multiplicative null model through the factorial form of the response ratio (Equation S2; Lajeunesse 2011), which has been shown to be statistically robust (Burgess et al 2022).…”

“…In contrast, if confidence intervals for an effect size do not overlap zero, then this interaction will be assigned a classification of either antagonistic, synergistic, or reversal. The method for classifying interactions within the classify_interactions function is adopted from the methodology outlined by Burgess et al (2022). An example of R code for the classify_interactions function is shown in Figure 6.…”

“…and reversals interactions have frequently been combined into a single classification. A detailed description of all four interaction classifications is available elsewhere (see Burgess et al 2022;Jackson et al 2016;Orr et al 2020); however, the classify_interactions function provides users with the option to specify whether reversal and antagonistic interactions should be differentiated through the assign_reversals argument (see Table 2, Figure 6).…”

“…Additionally, for the additive null model, it is important to note that sample sizes play a prominent role in the calculation of the effect size variance (Figure 8d) and hence effect size standard error (Figure 8e). Accordingly, under commonplace sample sizes for multiple stressor experiments (see Burgess et al 2022), calculations of effect size variance may be driven by sample size. This is evidenced by the various 'bands' in Figures 8d and 8e which correspond to mean treatment sample sizes of three, four, five, six, seven, and eight.…”

“…The multiplestressR R package allows researchers to implement either the additive (Gurevitch et al 2000) or multiplicative (Lajeunesse 2011) null models. Importantly, these forms of the null models have been implemented within previous multiple stressor research (e.g., Burgess et al 2021;Harvey et al 2013;Hillebrand 2002) and have been shown to be statistically rigorous (Burgess et al 2022).…”