Multimodal Word Meaning Induction From Minimal Exposure to Natural Text

Abstract

By the time they reach early adulthood, English speakers are familiar with the meaning of thousands of words. In the last decades, computational simulations known as distributional semantic models (DSMs) have demonstrated that it is possible to induce word meaning representations solely from word co-occurrence statistics extracted from a large amount of text. However, while these models learn in batch mode from large corpora, human word learning proceeds incrementally after minimal exposure to new words. In this study, we run a set of experiments investigating whether minimal distributional evidence from very short passages suffices to trigger successful word learning in subjects, testing their linguistic and visual intuitions about the concepts associated with new words. After confirming that subjects are indeed very efficient distributional learners even from small amounts of evidence, we test a DSM on the same multimodal task, finding that it behaves in a remarkable human-like way. We conclude that DSMs provide a convincing computational account of word learning even at the early stages in which a word is first encountered, and the way they build meaning representations can offer new insights into human language acquisition.

Symbol Grounding Without Direct Experience: Do Words Inherit Sensorimotor Activation From Purely Linguistic Context?

“…This is an interesting result by itself, as it demonstrates that learning about features of a novel concept can be achieved by presenting the respective novel words in the context of natural sentences, thereby extending the findings by Ouyang et al (2017) to another, more naturalistic learning scenario. Remarkably, this learning occurs even if the novel words do not directly correspond to an actual concept with which the learner already has experience, but they are instead textual chimeras created from two category members (Lazaridou et al, 2017). This result supports the distributional hypothesis that word meanings are at least partially determined by the linguistic context the words occur in (Harris, 1954;Landauer & Dumais, 1997;Lund & Burgess, 1996;Sahlgren, 2008).…”

“…However, our learning phases were also not shorter than those in similar studies that found learning effects on variables such as categorization performances and similarity judgements (Lazaridou et al., ; Ouyang et al., ). This, in combination with the fact that participants were able to make correct judgements in the explicit judgement task, leads us to the assumption that the learning phases we employed were long enough to form a representation for the novel words that is clear enough to perform a variety of different tasks (Lazaridou et al., ; Ouyang et al., ). In fact, in none of the learning phases did the participants get any explicit information about the vertical location of the novel words they learned, but only encountered them in linguistic contexts implying a certain vertical location.…”

“…However, the learning phases in our experiments took between 20 and 30 min to complete and were not shorter than the learning phase employed by € Ottl et al (2017), albeit qualitatively very different. However, our learning phases were also not shorter than those in similar studies that found learning effects on variables such as categorization performances and similarity judgements (Lazaridou et al, 2017;Ouyang et al, 2017). This, in combination with the fact that participants were able to make correct judgements in the explicit judgement task, leads us to the assumption that the learning phases we employed were long enough to form a representation for the novel words that is clear enough to perform a variety of different tasks (Lazaridou et al, 2017;Ouyang et al, 2017).…”

“…This result supports the distributional hypothesis that word meanings are at least partially determined by the linguistic context the words occur in (Harris, 1954;Landauer & Dumais, 1997;Lund & Burgess, 1996;Sahlgren, 2008). At least semantic features of novel words can be induced from such contexts, even with rather small amounts of learning experience for each novel word (Lazaridou et al, 2017).…”

“…As already mentioned, we replaced real words in natural text with novel words in order to create the material for this experiment. More specifically, we made use of textual chimeras (Lazaridou et al., ), where two different, but related, real words were replaced with a single novel word. For example, if Seige replaced both Mond ( moon ) and Wolke ( cloud ), it will most likely describe a large object far up in the sky.…”