Don't let spurious accusations of pseudoreplication limit our ability to learn from natural experiments (and other messy kinds of ecological monitoring)

Abstract

Pseudoreplication is defined as the use of inferential statistics to test for treatment effects where treatments are not replicated and/or replicates are not statistically independent. It is a genuine but controversial issue in ecology particularly in the case of costly landscape‐scale manipulations, behavioral studies where ethics or other concerns may limit sample sizes, ad hoc monitoring data, and the analysis of natural experiments where chance events occur at a single site. Here key publications on the topic are reviewed to illustrate the debate that exists about the conceptual validity of pseudoreplication. A survey of ecologists and case studies of experimental design and publication issues are used to explore the extent of the problem, ecologists’ solutions, reviewers’ attitudes, and the fate of submitted manuscripts. Scientists working across a range of ecological disciplines regularly come across the problem of pseudoreplication and build solutions into their designs and analyses. These include carefully defining hypotheses and the population of interest, acknowledging the limits of statistical inference and using statistical approaches including nesting and random effects. Many ecologists face considerable challenges getting their work published if accusations of pseudoreplication are made – even if the problem has been dealt with. Many reviewers reject papers for pseudoreplication, and this occurs more often if they haven't experienced the issue themselves. The concept of pseudoreplication is being applied too dogmatically and often leads to rejection during review. There is insufficient consideration of the associated philosophical issues and potential statistical solutions. By stopping the publication of ecological studies, reviewers are slowing the pace of ecological research and limiting the scope of management case studies, natural events studies, and valuable data available to form evidence‐based solutions. Recommendations for fair and consistent treatment of pseudoreplication during writing and review are given for authors, reviewers, and editors.

Contrasting multi-taxa diversity patterns between abandoned and non-intensively managed forests in the southern Dolomites

“…However, our findings can contribute to the formulation of new research hypotheses and to provide valuable data for meta-analyses, which is an increasingly used tool to explore general patterns in ecology (Davies & Gray 2015). Moreover, the results of this study can assist in the scientific assessment of adverse effects of forest management (Sitzia et al 2016), forest ecosystem functions mapping (Vizzarri et al 2015), and diagnostic species selection (Carranza et al 2012) in forest habitats.…”

Ocean warming and tropical invaders erode the performance of a key herbivore

“…We are aware that the use of "pseudoreplicates" is unorthodox; however, we presume that in this study this approach did not have any effect on the results presented or on their corresponding interpretations, as precautionary measures were taken (Oksanen 2001, Sandoval-Gil et al 2012, Davies and Gray 2015, Haddaway and Verhoeven 2015. We are aware that the use of "pseudoreplicates" is unorthodox; however, we presume that in this study this approach did not have any effect on the results presented or on their corresponding interpretations, as precautionary measures were taken (Oksanen 2001, Sandoval-Gil et al 2012, Davies and Gray 2015, Haddaway and Verhoeven 2015.…”

Artificial Light at Night Affects Organism Flux across Ecosystem Boundaries and Drives Community Structure in the Recipient Ecosystem

“…with complex questions at the ecosystem scale, as effort and cost associated with such large-scale experiments limits the use of replicated units (Carpenter, 1990(Carpenter, , 1996Skelly and Kiesecker, 2001;Davies and Gray, 2015;Barley and Meeuwig, 2017). It is thus possible that the effects observed here are the result of one or more factors other than the treatment, although we tried to capture potential effects by careful monitoring of important environmental variables (see Holzhauer et al, 2015).…”