What is our power to detect device effects in animal tracking studies?

Abstract

The use of bio‐logging devices to track animal movement continues to grow as technological advances and device miniaturisation allow researchers to study animal behaviour in unprecedented detail. Balanced against the remarkable data that bio‐loggers can provide is a need to understand the impact of devices on animal behaviour and welfare. Recent meta‐analyses have demonstrated the impacts of device attachment on animal behaviour, but there is a concern about the frequency and clarity with which device effects are reported. One aspect lacking in many studies is assessment of the statistical power of tests of device effects, yet such information would assist the interpretation of results. We address this issue by providing an overview of the statistical power, as well as the Type M (magnitude) and Type S (sign) error rate, of tests of device effects within the avian tracking literature across a range of assumed effect sizes. The median power of statistical tests ranged from 9% to 65% across a range of assumed effect sizes corresponding to benchmark values for small, moderate and large effects (d = 0.2, 0.5, 0.8, respectively). Moreover, when using effect sizes derived from previous a meta‐analysis (d = 0.1), median power was only 6%. When assuming smaller effect sizes, statistical tests were characterised by high Type M and Type S error rates, suggesting that statistically significant results of device effects will tend to exaggerate the size of such effects and may estimate the sign of an effect in the wrong direction. Well‐designed tracking studies will reduce device effects to low levels and consequently issues associated with low power will be commonplace. Nevertheless, assessment of device effects remains important, particularly when embarking on novel tracking studies. We recommend that statistical tests of device effects are reported clearly and are routinely accompanied by assessment of statistical power, including Type M and Type S errors, based upon realistic external estimates of effect size. Reporting the statistical power can help avoid the pitfalls of overstating results from individual studies, shift the emphasis to accurate reporting of effect sizes and guide decisions about the ethical impacts of device attachment.

Low statistical power and overestimated anthropogenic impacts, exacerbated by publication bias, dominate field studies in global change biology

“…Recently, a few papers have pointed out the importance of quantifying the Type M and S error rates (Cleasby et al, 2021;Lemoine et al, 2016;. For example, Lemoine et al (2016) showed that reported effect sizes of global warming on plant and non-manipulative observations had very similar results in terms of their power, Type M and S errors.…”

“…Recently, a few papers have pointed out the importance of quantifying the Type M and S error rates (Cleasby et al, 2021 ; Lemoine et al, 2016 ; T. H Parker et al, 2018 ). For example, Lemoine et al ( 2016 ) showed that reported effect sizes of global warming on plant growth were, on average, three times larger than a “true” effect that was approximated by an overall meta‐analytic mean (Type M error rate: 3).…”

The impact of bio-logging on body weight change of the Eurasian beaver

“…We found a negative effect on beaver body weight change during the tagging period, but with considerable individual variation amongst tagged individuals. These findings might have implications regarding animal welfare and the validity of data collected [ 103 , 104 ]. The large variation among tagged beavers illustrates how effects of tagging should not only be studied by comparing averages among tagged and untagged individuals, but also by analyzing individual impacts.…”

Foraging distribution of breeding northern fulmars is predicted by commercial fisheries

“…3.2%). Despite these considerations, we acknowledge that negative effects remain difficult to detect, control and quantify and could still lead to behavioural ano malies ( Vandenabeele et al 2014, Cleasby et al 2021.…”