Bayesian sample size planning for developmental studies

Abstract

Running developmental experiments, particularly with infants, is often time‐consuming and intensive, and the recruitment of participants is hard and expensive. Thus, an important goal for developmental researchers is to optimize sampling plans such that neither too many nor too few participants are tested given the hypothesis of interest. One approach that enables such optimization is the use of Bayesian sequential designs. The use of such sequential designs allows data collection to be terminated as soon as the evidence is deemed sufficiently strong, without compromising the interpretability of the test outcome. In this tutorial, we illustrate how to plan a Bayesian sequential testing design prior to data collection by the method of Bayes factor design analysis—the Bayesian equivalent of power analysis—and discuss the relevance of this for developmental psychologists. The tutorial provides a step‐by‐step guide to perform such analyses, and the methods are illustrated using commonly used statistics in a typical infant‐looking time paradigm such that researchers can easily adapt these methods for their studies.Highlights Bayesian Sequential Testing can be used to optimize sample sizes and save on data collection. Bayes Factor Design Analysis can be used to analyze a sequential testing study prior to data collection. Step‐by‐step guide for performing Bayes Sequential Testing and Bayes Factor Design Analysis.

Habituation, part II. Rethinking the habituation paradigm

“…However, issues with low power are of course influenced by various design choices, other than the habituation criteria alone. These design choices that affect power are out of the scope of this article (DeBolt et al, 2020;Oakes, 2017;Visser et al, 2021).…”

“…Further, with more complex designs and statistical models, the required sample sizes may become too large. A common approach to this problem would be to first conduct studies of descriptive nature using rules of thumb for planning sample sizes, with the hope that such previous studies would provide enough information to plan future studies with more care, or to inform the statistical models with prior knowledge (Visser et al, 2021). An alternative approach would be to conduct sequential designs: Data collection is continued until a sufficient amount of evidence about the question of interest accumulates, or the researcher runs out of resources or patience (Schönbrodt et al, 2017;Stefan et al, 2019).…”

“…Sequential F I G U R E 7 Plot of real data from Bergmann and Cristia (2018). Points highlighted red are trials that are close to the maximum trial length designs hold promising potential to improve the efficiency of experiments, and as such are valuable especially in contexts where achieving large sample sizes is challenging or unlikely (Visser et al, 2021). Evidence from different studies may be integrated to plan new experiments, which would further improve the study design efficiency over the long run (Stefan et al, 2019).…”

Open science and metascience in developmental psychology: Introduction to the special issue

“…Turoman et al (2022) present a workflow for applying open science principles in a developmental psychology lab, using their own lab as an example. Regarding data analysis, Visser et al (2023) present a tutorial for using Bayesian sequential testing designs and Woods et al (2023) present best practices for addressing missing data through multiple imputations.…”