Is Reliability of Cognitive Measures in Children Dependent on Participant Age? A Case Study with Two Large-Scale Datasets

Abstract

When assessing children in laboratory experiments, the measured responses also contain task-irrelevant participant-level variability (“noise”) and measurement noise. Since experimental data are used to make inferences of development of cognitive capabilities with age, it is important to know if reliability of the used measurements depends on child age. Any systematic age-dependent changes in reliability could result in misleading developmental trajectories, as lower reliability will necessarily result in smaller effect sizes. This paper examines age-dependency of task-independent measurement variability in early childhood (3–40 months) by analyzing two large-scale datasets of participant-level experimental responses: the ManyBabies infant-directed speech preference (MB-IDS) dataset and a saccadic reaction time (SRT) dataset collected from rural South Africa. Analysis of participant- and study-level data reveals that MB-IDS shows comparable reliability across the included age range. In contrast, SRTs reflect systematically increasing measurement consistency with increasing age. Potential reasons and implications of this divergence are briefly discussed.

Introducing Meta‐analysis in the Evaluation of Computational Models of Infant Language Development

“…In other words, infant effect sizes are inherently smaller than the "true effect" reflecting stimulus processing only, whereas model effect sizes are accurate reflections of model capability. In addition, there is no known technique to correct for the underestimation of infant effect sizes, as the amount of noise in human data is generally unknown, varies from one study and paradigm to another, and is likely to be smaller with older infants and experimental paradigms applicable to them (but see Räsänen, Cruz Blandón, & Leppänen (2023) for a recent attempt to evaluate age-dependency of measurement reliability). Hence, the upper bound of the human effect size magnitude is not known.…”