Guidelines for estimating repeatability

Abstract

Summary 1.Researchers frequently take repeated measurements of individuals in a sample with the goal of quantifying the proportion of the total variation that can be attributed to variation among individuals vs. variation among measurements within individuals. The proportion of the variation attributed to variation among individuals is known as repeatability and is most frequently estimated as the intraclass correlation coefficient (ICC). The goal of our study is to provide guidelines for determining the sample size (number of individuals and number of measurements per individual) required to accurately estimate the ICC. 2. We report a range of ICCs from the literature and estimate 95% confidence intervals for these estimates. We introduce a predictive equation derived by Bonett (2002), and we test the assumptions of this equation through simulation. Finally, we create an R statistical package for the planning of experiments and estimation of ICCs. 3. Repeatability estimates were reported in 1AE5% of the articles published in the journals surveyed. Repeatabilities tended to be highest when the ICC was used to estimate measurement error and lowest when it was used to estimate repeatability of behavioural and physiological traits. Few authors report confidence intervals, but our estimated 95% confidence intervals for published ICCs generally indicated a low level of precision associated with these estimates. This survey demonstrates the need for a protocol to estimate repeatability. 4. Analysis of the predictions from Bonett's equation over a range of sample sizes, expected repeatabilities and desired confidence interval widths yields both analytical and intuitive guidelines for designing experiments to estimate repeatability. However, we find a tendency for the confidence interval to be underestimated by the equation when ICCs are high and overestimated when ICCs and the number of measurements per individual are low. 5. The sample size to use when estimating repeatability is a question pitting investigator effort against expected precision of the estimate. We offer guidelines that apply over a wide variety of ecological and evolutionary studies estimating repeatability, measurement error or heritability. Additionally, we provide the R package, icc, to facilitate analyses and determine the most economic use of resources when planning experiments to estimate repeatability.

Protein and carbohydrate intake influence sperm number and fertility in male cockroaches, but not sperm viability

“…In a subset of 20 males, our estimate of sperm number was shown to be highly repeatable using this sampling procedure (repeatability ¼ 0.93, 95% CI: 0.89, 0.97) [46]. The mean number of sperm across these subsamples was used in all subsequent analyses.…”

Broad thermal tolerance is negatively correlated with virulence in an opportunistic bacterial pathogen

“…Consequently, we chose to use maximum yield as a robust measure of strain performance at a given temperature (two measurements per temperature per strain). For each temperature, the repeatability of maximum yield measurement (intraclass correlation coefficient (ICC)) was defined as R  =  V G /( V G  +  V R ), where V G is the variance among strains and V R is the variance within strains (Sokal & Rohlf, 1995; Wolak, Fairbairn, & Paulsen, 2012). Repeatability was calculated using the rptR package for R (Stoffel, Nakagawa, & Schielzeth, 2017).…”

Evaluation of an on-farm method to assess colostrum IgG content in sows

“…Brix percentage repeatability was assessed with the intraclass correlation coefficient (ICC). Data are highly considered reproducible when ICC > 0.8 (Wolak et al, 2012). The storage temperature and length effect on colostrum IgG were analyzed using one-way repeatedmeasures ANOVA.…”