Eye movements reveal persistent sensitivity to sound symbolism during word learning.

Abstract

Although the relationship between sound and meaning in language is assumed to be largely arbitrary, reliable correspondences between sound and meaning in natural language appear to facilitate word learning. Using a set of independently normed pseudoword and shape stimuli, we examined the real-time effects of sound-to-shape correspondences at initial presentation and throughout an extended learning process resulting in high accuracy. In addition to accuracy and response time (RT) measures, we monitored participants' eye movements to investigate the extent to which visual orienting to objects is influenced by the sound symbolic characteristics of novel labels at initial exposure and throughout learning. Over the course of word learning, congruency of sound and shape properties affected both accuracy and RT with higher accuracy and faster responses for congruent than incongruent items. Eye tracking data reveal that congruent targets were fixated faster than incongruent targets throughout learning and that nontargets consistent with the sound symbolic properties of the word remained attractive distracters, even after overt behavioral differences in accuracy disappeared. This demonstrates the sustained influence of sound symbolism and the importance of sensitive, continuous measures of assessing sound symbolic effects in word learning and lexical processing. Arbitrariness resulted in better final individuation performance only when the arbitrary items were more phonologically distinct than the sound symbolic stimuli. These findings suggest that the advantages of sound symbolism may persist beyond early word learning and serve to significantly influence online lexical processing. (PsycINFO Database Record

Acoustic and Visual Stimulus Parameters Underlying Sound Symbolic Crossmodal Correspondences

“…Studies of sound-symbolic CCs also show that they are important for determining meaning across general categories. The findings from Gasser (2004) also indicate that the advantage of determining the specific meanings of words using sound symbolism is diminished as the vocabulary increases (although see Revill et al, 2018). Whether sound symbolism facilitates learning categories or specific word meanings in adults, the idea that sound symbolism can benefit adult learners has been recently validated with eye-tracking and other behavioral studies (Brand et al, 2018;Revill et al, 2018).…”

“…For example, individuals exhibit sensitivity to sound-symbolic crossmodal associations as early as four months of age (Ozturk et al, 2013), and recent studies have suggested that sound symbolism is important for specific word-to-meaning associations in young children with limited vocabularies (Gasser, 2004;Tzeng, Nygaard, & Namy, 2017). In adults, sound symbolism may offer linguistic processing advantages for categorization and word learning, particularly for larger vocabularies (Brand et al, 2018;Gasser, 2004;Revill et al, 2018). Though most of the evidence supporting sound symbolism has consisted of behavioral studies, more recently neuroimaging studies have begun to reveal the neural correlates of this phenomenon (Revill et al, 2014;McCormick et al, 2018;Peiffer-Smadja & Cohen, 2019).…”

“…the spectrotemporal features and amplitude variations of speech sounds and the spatial features of shapes) have received limited examination. In tasks that test artificial language learning, non-arbitrary word-meaning associations are easier to learn than arbitrarily constructed word-meaning associations (Brand et al, 2018;Nielsen, 2016;Revill et al, 2018), but researchers do not know what stimulus features participants attend to during the perception of these artificial words. Assessing the physical features that contribute to a sound-symbolic CC, as we do here, provides insight into the fundamental perceptual processes that drive word-tomeaning associations.…”

A Developmental Approach to Machine Learning?

“…For example, if a 2-year-old child encounters their very first tractor – say, a green John Deere working in a field – while hearing its name, the child from that point forward will recognize all variety of tractors as tractors – red Massey-Fergusons, antique tractors, ride-on mowers – but not backhoes or trucks. This phenomenon, known as the “shape bias” in the developmental literature is an example of “one-shot” learning that has been observed in children’s natural category learning and has been replicated and extensively studied in the laboratory (e.g., Rosch et al, 1976 ; Landau et al, 1988 ; Samuelson and Smith, 2005 ).…”

“…Around 18 to 24 months (with the timing different for different children), the character and rate of learning changes. Around 2 years of age, object name learning becomes seemingly effortless as typically developing children need very little experience, often just a single experience with a named object, to generalize the name appropriately to new instances ( Landau et al, 1988 ; Smith, 2003 ). The shift from slow incremental learning to rapid nearly “one-shot” learning reflects changes in the internal machinery brought on by learning itself ( Smith et al, 2002 ).…”