Do current statistical learning tasks capture stable individual differences in children? An investigation of task reliability across modality

Abstract

Do commonly used statistical-learning tasks capture stable individual differences in children? Infants, children, and adults are capable of using statistical learning (SL) to extract information about their environment. Although most studies have looked at group-level performance, a growing literature examines individual differences in SL and their relation to language-learning outcomes: Individuals who are better at SL are expected to show better linguistic abilities. Accordingly, studies have shown positive correlations between SL performance and language outcomes in both children and adults. However, these studies have often used tasks designed to explore group-level performance without modifying them, resulting in psychometric shortcomings that impact reliability in adults (Siegelman, Bogaerts, Christiansen, & Frost in Transactions of the Royal Society B, 372, 20160059, 2017a; Siegelman, Bogaerts, & Frost in Behavior Research Methods, 49, 418-432, 2017b). Even though similar measures are used to assess individual differences in children, no study to date has examined the reliability of these measures in development. This study examined the reliability of common SL measures in both children and adults. It assessed the reliability of three SL tasks (two auditory and one visual) twice (two months apart) in adults and children (mean age 8 years). Although the tasks showed moderate reliability in adults, they did not capture stable individual variation in children. None of the tasks were reliable across sessions, and all showed internal consistency measures well below psychometric standards. These findings raise significant concerns about the use of current SL measures to predict and explain individual differences in development. The article ends with a discussion of possible explanations for the difference in reliability between children and adults. Keywords Statistical learning. Individual differences. Reliability. Domain generality. Children Infants, children, and adults are constantly exposed to recurring patterns in their environment and manage to learn and generalize from them. This ability-often called statistical learning (SL)-is postulated to be one of the important mechanisms in language learning and in learning more generally (e.g., Erickson & Thiessen, 2015). Statistical learning in infants was demonstrated in a seminal study showing that 8month-old infants can use distributional information about syllable co-occurrence to discover word boundaries (Saffran, Aslin, & Newport, 1996). This study led to a surge in research probing this ability in both infants and adults. Research over the past 20 years has shown that statistical learning is present from early infancy (Bulf, Johnson, & Valenza, 2011), is found across modalities (visual, auditory, and tactile; Conway & Christiansen, 2005; Kirkham, Slemmer, & Johnson, 2002), and can be used to learn a range of linguistic properties (e.g., phonetic categories, word order, and phrase structure; see Romberg & Saffran, 2010, for a review). These studies, which as...

Statistical Learning in the Visuomotor Domain and Its Relation to Grammatical Proficiency in Children with and without Developmental Language Disorder: A Conceptual Replication and Meta-Analysis

“…Another potential explanation for our null result on the association between serial reaction time performance and grammatical proficiency may concern the reliability of the disruption peak as an individual measure of statistical learning in the serial reaction task (see Arnon, 2019;West et al, 2017 for discussions of the low psychometric properties of the currently available measures to assess individual differences in statistical learning). Our disruption peak measure had a split-half reliability of 0.62, with a 95% confidence interval ranging from 0.38 to 0.76.…”

Seven-year-olds recall non-adjacent dependencies after overnight retention

“…Further, we showed that an individual's stronger change from a positive ERP effect on day one to a negative ERP effect on day two was correlated with stronger positive behavioral changes (i.e., more correct answers on day two compared to day one). These results have to be interpreted with caution because individual differences in statistical learning show questionable reliability (Arnon, 2019;Siegelman et al, 2017aSiegelman et al, , 2017bWest et al, 2018) and the behavioral performance was not above chance at the group level. However, our results at the individual level are in line with our results at the group level.…”

“…Since these measures were acquired in the same task, their correlation should not be over-interpreted. Recently, several concerns have been raised about the reliability of offline measures of individual differences in statistical learning (Siegelman et al, 2017a(Siegelman et al, , 2017bWest et al, 2018;Arnon, 2019). In the present study, we address some of these concerns, as…”

“…Please note that our group-level analyses, on the other hand, are not affected by these concerns (seeArnon, 2019).…”

“…A number of recent studies have raised concerns that group-level offline tasks, which assess statistical learning, may not provide reliable measures of individual differences West, Vadillo, Shanks, & Hulme, 2018), particularly in children (Arnon, 2019). Siegelman, Bogaerts, Christiansen, and Frost (2017) suggest that online measures may circumvent some of the problems seen in the reliability of offline tasks.…”