Invited review: Reproducible research from noisy data: Revisiting key statistical principles for the animal sciences

Abstract

Reproducible results define the very core of scientific integrity in modern research. Yet, legitimate concerns have been raised about the reproducibility of research findings, with important implications for the advancement of science and for public support. With statistical practice increasingly becoming an essential component of research efforts across the sciences, this review article highlights the compelling role of statistics in ensuring that research findings in the animal sciences are reproducible-in other words, able to withstand close interrogation and independent validation. Statistics set a formal framework and a practical toolbox that, when properly implemented, can recover signal from noisy data. Yet, misconceptions and misuse of statistics are recognized as top contributing factors to the reproducibility crisis. In this article, we revisit foundational statistical concepts relevant to reproducible research in the context of the animal sciences, raise awareness on common statistical misuse undermining it, and outline recommendations for statistical practice. Specifically, we emphasize a keen understanding of the data generation process throughout the research endeavor, from thoughtful experimental design and randomization, through rigorous data analysis and inference, to careful wording in communicating research results to peer scientists and society in general. We provide a detailed discussion of core concepts in experimental design, including data architecture, experimental replication, and subsampling, and elaborate on practical implications for proper elicitation of the scope of reach of research findings. For data analysis, we emphasize proper implementation of mixed models, in terms of both distributional assumptions and specification of fixed and random effects to explicitly recognize multilevel data architecture. This is critical to ensure that experimental error for treatments of interest is properly recognized and inference is correctly calibrated. Inferential misinterpretations associated with use of P-values, both significant and not, are clarified, and problems associated with error inflation due to multiple comparisons and selective reporting are illustrated. Overall, we advocate for a responsible practice of statistics in the animal sciences, with an emphasis on continuing quantitative education and interdisciplinary collaboration between animal scientists and statisticians to maximize reproducibility of research findings.

Comparison of effects of four weaning methods on health and performance of beef calves

“…Nonetheless, it is informative to recognize outcomes may differ between two locations even when using similar management approaches. Perhaps most important, the demonstration of some differences between the two locations further bolsters the importance of findings that were consistent across both, as it demonstrates repeatability (Bello and Renter, 2018).…”

“…It is unclear why previous studies have yielded inconsistent or conflicting conclusions. Potential reasons include the use of relatively small groups (e.g., Burke et al (2009) utilized 36 steers, while Campistol et al (2016) enrolled 48); inadequate nutritional support to detect differences in performance (Haley et al, 2005); or use of a single herd/ source, which precludes in-study replication and may therefore hinder assessment of reproducibility (Bello and Renter, 2018). Regardless of cause, the general lack of repeatability makes recommendations difficult.…”

Investigating causal biological relationships between reproductive performance traits in high-performing gilts and sows1

“…This approach is consistent with Dórea et al, (2018), who used a machine-learning strategy to leave-one-trial-out of the dataset at a time when validating feed intake predictions in lactating dairy cows. So adapted, Jackknife resampling might be considered a validation strategy in a broader scope of inference (Bello and Renter, 2018) across the population for which the body weight blocks used in this study [or trials used by Dórea et al (2018)] might be considered a representative, if not random, sample.…”

Generalized additive mixed model on the analysis of total transport losses of market-weight pigs1

“…Mixed models have the benefit of accommodating random effects into the statistical analysis, thereby accounting for the hierarchical organization of the data, which is naturally inherited from the data collection process. Taking into consideration, the structure of the data allows the partition of the total random variation across multiple levels of the data architecture, such as farms and transport companies, providing a more calibrated inference for hypothesis testing (Tempelman, 2009;Bello and Renter, 2018). In this study, we treated the combination of sites and quarters of the year, as well as truck companies as random effects, and they represented 21.2% and 0.59% of the total random variation of TTL, respectively.…”