Proceedings of the Workshop on Dimensions of Meaning: Distributional and Curated Semantics (DistCurate 2022)

Abstract

Present-day language models provide contextual representations of words: a given element of the model's vocabulary will generally be represented differently depending on the larger (linguistic and/or non-linguistic) context in which it appears. This is undoubtedly a key feature of the success of these models. What do these contextual representations mean for linguists working on lexical semantics? In this talk, I'll argue, first and foremost, that contextual representations are better aligned with linguists' conception of word meaning than any previous representation scheme in NLP, including symbolic approaches. I will then describe some ways in which linguists can use large pretrained language models to gain new insights into lexical meaning, and some ways in which language model development could be fruitfully informed by findings in linguistics. The overall take-away is that the turn toward contextual representations has created an exciting new space for collaboration between linguists and NLPers.Bio: Christopher Potts is Professor and Chair of Linguistics and Professor (by courtesy) of Computer Science at Stanford, and a faculty member in the Stanford NLP Group and the Stanford AI Lab. His group uses computational methods to explore topics in emotion expression, context-dependent language use, systematicity and compositionality, and model interpretability. This research combines methods from linguistics, cognitive psychology, and computer science, in the service of both scientific discovery and technology development. He is the author of the 2005 book The Logic of Conventional Implicatures as well as numerous scholarly papers in computational and theoretical linguistics.