Biases in studies of spatial patterns in insect herbivory

Abstract

The properties of the human mind are responsible for a number of biases that affect the quality of scientific research. However, scientists working in the fields of ecology and environmental science rarely take these biases into account. We conducted a meta‐analysis of data extracted from 125 publications comparing woody plant damage by defoliating insects in different environments in order to understand the extent to which our knowledge on spatial patterns in herbivory is affected by various biases. We asked which research methods are most prone to biases and whether these biases lead to overestimation of the effects under study. The effect sizes (ESs) decreased with increases in the numbers of plant species involved in the study, with 61% lower ESs for herbivory estimated on all plants growing in study plots compared to herbivory on selected species. ESs also depended on the leaf sampling procedure: when all leaves from a tree or branch were sampled for measurements of herbivory or when random or systematic selection protocols were applied, ESs were 74% smaller than in cases of more subjective haphazard selection. In addition, ESs were 97% and 135% greater when the person conducting sampling and measuring was aware of the research hypothesis or sample origin, when compared with situations when the observer was blinded to these factors. The impacts of cognitive biases on the study outcomes significantly decreased with the increase in publication year; however, this pattern emerged mostly due to high‐ranked journals and was non‐significant for other journals. Using the studies of spatial patterns in herbivory as an example, we showed that our ecological and environmental knowledge is considerably biased due to an unconscious tendency of researchers to find support for their hypotheses and expectations, which generally leads to overestimation of the effects under study. Cognitive biases can be avoided by using different methods, such as applying randomization procedures in sampling and blinding of research hypotheses and sample origins. These measures should be seen as obligatory; otherwise, accumulation of the biased results in primary studies may ultimately lead to false general conclusions in subsequent research synthesis.

Can School Children Support Ecological Research? Lessons from the <i>Oak Bodyguard</i> Citizen Science Project

“…Finally, whenever possible, the researcher analyzing the data should recover the raw material collected by children, or at the very least access pictures that allow for the re-assessment of measurements (Ekholm et al 2019;Steinke et al 2017). Importantly, these recommendations also hold true for large multi-partner research programs, as we also detected bias in data collected by professional scientists (Zvereva and Kozlov 2019). Whether variability in observations made by schoolchildren is random or can be modelled using appropriate covariates is an important question deserving further attention.…”

“…It is possible that schoolchildren (and their teachers too) felt they had to see predation marks because this is what they perceived as the aim of the experiment. However, although confirmation bias is more likely to occur in schoolchildren and their teachers, it is important to stress that this type of cognitive bias is also common among trained professional scientists who may have interpreted small cracks (for example) on the caterpillar surface as predation marks (Forstmeier, Wagenmakers, and Parker 2017;Zvereva and Kozlov 2019). Although the protocol clearly specified how to standardize caterpillar size and shape, and emphasized the importance of standardization, we noticed that the dimensions of dummy caterpillars varied widely, both within and among schools.…”

Hiding in the background: community-level patterns in invertebrate herbivory across the tundra biome

“…These studies however, were investigating herbivory levels on a single or a few plant species rather than at the community level. The number of plant species involved in studies estimating herbivore damage can affect the inferences of these studies, with studies including fewer species tending to overestimate damage (Zvereva and Kozlov 2019). Warming can also influence the feeding choices of invertebrate herbivores (Barrio et al 2016a, Gamarra et al 2018, so patterns of herbivory of a single species may not be representative of what happens at the community level.…”

Can school children support ecological research? Lessons from the‘Oak bodyguard’citizen science project

“…Finally, whenever possible, the researcher analyzing the data should recover the raw material collected by children, or at the very least pictures allowing the re-assessment of measurements 12,29 . Importantly, these recommendations also hold true for large multipartners research programs, as we also detected bias in data collected by professional scientists 34 . Whether variability in observations made by school children is random or can be modelled using appropriate covariates is an important question deserving further attention.…”

“…However, although confirmation bias is more likely to occur in school children and their teachers, it is important to stress that this type of cognitive bias is also common among trained professional scientists, who may have interpreted e.g. small cracks on caterpillar surface as predation marks 34,35 .…”