Finding Structure in One Child's Linguistic Experience

Abstract

Neural network models have recently made striking progress in natural language processing, but they are typically trained on orders of magnitude more language input than children receive. What can these neural networks, which are primarily distributional learners, learn from a naturalistic subset of a single child's experience? We examine this question using a recent longitudinal dataset collected from a single child, consisting of egocentric visual data paired with text transcripts. We train both languageonly and vision-and-language neural networks and analyze the linguistic knowledge they acquire. In parallel with findings from Jeffrey Elman's seminal work, the neural networks form emergent clusters of words corresponding to syntactic (nouns, transitive and intransitive verbs) and semantic categories (e.g., animals and clothing), based solely on one child's linguistic input. The networks also acquire sensitivity to acceptability contrasts from linguistic phenomena, such as determiner-noun agreement and argument structure. We find that incorporating visual information produces an incremental gain in predicting words in context, especially for syntactic categories that are comparatively more easily grounded, such as nouns and verbs, but the underlying linguistic representations are not fundamentally altered. Our findings demonstrate which kinds of linguistic knowledge are learnable from a snapshot of a single child's real developmental experience.

The roles of neural networks in language acquisition

“…The proof-of-concept approach can model how in practice some linguistic outcome can be learnt, offering support for these types of hypotheses. This approach has been one of the most fruitful in recent years and has been used to support multiple hypotheses about the learnability of : the shape bias (Ohmer, Marino, König, & Franke, 2021), basic syntactic dependencies (Huebner et al, 2021), syntactic and semantic categories (Wang, Vong, Kim, & Lake, 2023), associative word learning strategies (Vong & Lake, 2022), words from sustained attention (Tsutsui, Chandrasekaran, Reza, Crandall, & Yu, 2020), and logical reasoning words (Portelance, Frank, & Jurafsky, 2023).…”

Acquiring Linguistic Knowledge from Multimodal Input

“…Lazaridou et al (2017) and Chrupała et al (2015) are notable for pioneering self-supervised training objectives for multimodal models several years before the advent of Transformer architectures trained on masking objectives. Wang et al (2023) train LMs on data from the SAYCam dataset (Sullivan et al, 2021), pairing (written) child-directed utterances with visual data from the child's point of view. While this data domain is nearly ideal from a developmental plausibility perspective, the available data is too small to model anything past the first month of development.…”