Accumulating evidence in ecology: Once is not enough

Abstract

Many published studies in ecological science are viewed as stand‐alone investigations that purport to provide new insights into how ecological systems behave based on single analyses. But it is rare for results of single studies to provide definitive results, as evidenced in current discussions of the “reproducibility crisis” in science. The key step in science is the comparison of hypothesis‐based predictions with observations, where the predictions are typically generated by hypothesis‐specific models. Repeating this step allows us to gain confidence in the predictive ability of a model, and its corresponding hypothesis, and thus to accumulate evidence and eventually knowledge. This accumulation may occur via an ad hoc approach, via meta‐analyses, or via a more systematic approach based on the anticipated evolution of an information state. We argue the merits of this latter approach, provide an example, and discuss implications for designing sequences of studies focused on a particular question. We conclude by discussing current data collection programs that are preadapted to use this approach and argue that expanded use would increase the rate of learning in ecology, as well as our confidence in what is learned.

Assessing trends and vulnerabilities in the mutualism between whitebark pine (Pinus albicaulis) and Clark’s nutcracker (Nucifraga columbiana) in national parks of the Sierra-Cascade region

“…Alternative metrics and alternative models should be considered, however, and long-term monitoring under the NPS Vital Signs program can support this goal. Data from monitoring programs can be used to discriminate among our hypotheses and others in an iterative process of model updating and accumulation of evidence to advance our understanding and guide resource management [81]. Finally, the spatial scale of monitoring should match the vagility of the seed predator.…”

Modeling How Community Assembly Alters the Functioning of Ecosystems

“…Our study also illustrates that one can discriminate between a-priori hypotheses and ad-hoc hypotheses in ways that relate to the entire issue of ‘replicability’ in science (e.g. Nichols et al 2019). Here we set out to test 8 different hypotheses about how resource competition affects the relations between community assembly and the functioning of ecosystems.…”

Make like a glass frog: In support of increased transparency in herpetology

“…Following the recommendations of TOP, journals must be open to publishing replications and negative results (Nakagawa & Parker, 2015), dispensing with statements encouraging novelty. If journals can relieve the pressure for positive/significant results, the incentives to undertake questionable practices will cease to exist (Nilsen et al, 2020), and counter publication biases (Nichols et al, 2019). Replications do not suffer the lower citations rates that journals fear (Forstmeier et al, 2017), and researchers welcome them (Fraser et al, 2019).…”

“…• Include spatial and temporal scale of study (i.e., study site and plot extents, study duration -be as specific as possible) • Provide clear details on how means/medians are calculated from the total sample or subsets • Report sample size and confidence/credible intervals for each statistical analysis • Report all results regardless of outcome/significance • Further suggestions can be found in Fidler et al (2018) and Gerstner et al 2017By following best practice for reporting we can maximise studies' utility and thus optimise for metaanalysis inclusion (Hillebrand & Gurevitch, 2013;Nichols et al, 2019). Ensuring that methods and statistics are fully reported boosts reach and citations (Gerstner et al, 2017).…”