Neural reality of argument structure constructions

Abstract

In lexicalist linguistic theories, argument structure is assumed to be predictable from the meaning of verbs. As a result, the verb is the primary determinant of the meaning of a clause. In contrast, construction grammarians propose that argument structure is encoded in constructions (or form-meaning pairs) that are distinct from verbs. Decades of psycholinguistic research have produced substantial empirical evidence in favor of the construction view. Here we adapt several psycholinguistic studies to probe for the existence of argument structure constructions (ASCs) in Transformerbased language models (LMs). First, using a sentence sorting experiment, we find that sentences sharing the same construction are closer in embedding space than sentences sharing the same verb. Furthermore, LMs increasingly prefer grouping by construction with more input data, mirroring the behaviour of non-native language learners. Second, in a "Jabberwocky" priming-based experiment, we find that LMs associate ASCs with meaning, even in semantically nonsensical sentences. Our work offers the first evidence for ASCs in LMs and highlights the potential to devise novel probing methods grounded in psycholinguistic research.

CxGBERT: BERT meets Construction Grammar

“…They find, through manual analysis, that while most of these resultant groupings are indeed constructional, some of them might be too simplistic. Li et al (2022) similarly explore the extent to which language models have access to constructional information and more specifically, argument structure constructions using stimuli generated from templates. The authors thus adapt several psycholinguistic studies to Transformer-based language models.…”

“…In an effort to simulate language acquisition (through language exposure), Li et al (2022) use different language sample sizes and find that the more input the models get, the more likely they are to group sentences together based on their shared constructional pattern rather than their shared verb. Notably they show that RoBERTa (Y. , for example, seems to generalize meaning without lexical overlap from different constructions.…”

“…That being said, we have also shown that very advanced models and methods can find construction candidates in large amounts of data. For example, Li et al (2022) have shown how models are capable of identifying and learning constructions and Dunn's induction algorithm has successfully identified some serious candidate constructions. Moreover, incorporating techniques like Abstract Meaning Representation (AMR; Knight et al 2021) holds the potential to further enhance these capabilities.…”