Language (Re)modelling: Towards Embodied Language Understanding

Tamari, Ronen; Shani, Chen; Hope, Tom; Petruck, Miriam R. L.; Abend, Omri; Shahaf, Dafna

doi:10.18653/v1/2020.acl-main.559

Cited by 11 publications

(10 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Li et al [208] proposed a MIND (Mental Imagery eNhanceD) module to model the dynamics of the environment and create objects for a better understanding of the implemented agent. Tamari et al [209] described that natural language in cognitive linguistics (ECL) is inherently executable and driven by metaphorical mappings and mental simulations to schemas learned through hierarchical organization and interaction.…”

Section: ) Simulationmentioning

confidence: 99%

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

2022

View full text Add to dashboard Cite

Unlike previous studies on the Metaverse based on Second Life, the current Metaverse is based on the social value of Generation Z that online and offline selves are not different. With the technological development of deep learning-based high-precision recognition models and natural generation models, Metaverse is being strengthened with various factors, from mobile-based always-on access to connectivity with reality using virtual currency. The integration of enhanced social activities and neural-net methods requires a new definition of Metaverse suitable for the present, different from the previous Metaverse. This paper divides the concepts and essential techniques necessary for realizing the Metaverse into three components (i.e., hardware, software, and contents) and three approaches (i.e., user interaction, implementation, and application) rather than marketing or hardware approach to conduct a comprehensive analysis. Furthermore, we describe essential methods based on three components and techniques to Metaverse's representative Ready Player One, Roblox, and Facebook research in the domain of films, games, and studies. Finally, we summarize the limitations and directions for implementing the immersive Metaverse as social influences, constraints, and open challenges.

show abstract

Section: ) Simulationmentioning

confidence: 99%

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

2022

View full text Add to dashboard Cite

show abstract

“…Third, as others have noted (Bender and Koller, 2020;Bisk et al, 2020;Tamari et al, 2020;Borghi et al, 2019), grounding goes beyond sensorimotor associations. Linguistic meaning is also grounded in social experience and interaction.…”

Section: Limitationsmentioning

confidence: 88%

“…Most large language models (LMs) are trained on linguistic input alone. This approach may be fundamentally limited when it comes to language understanding (Bender and Koller, 2020;Bisk et al, 2020;Tamari et al, 2020), as the meaning of a word arguably depends on factors beyond which words it co-occurs with. In particular, humans appear to ground a word's meaning in a rich network of sensorimotor associations (Pulvermüller, 1999;Bergen and Feldman, 2008;Barsalou, 1999;Winter and Bergen, 2012;Barsalou, 2008;Glenberg and Kaschak, 2002).…”

Section: Introductionmentioning

confidence: 99%

Contextualized Sensorimotor Norms: multi-dimensional measures of sensorimotor strength for ambiguous English words, in context

Trott¹,

Bergen²

2022

Preprint

View full text Add to dashboard Cite

Most large language models are trained on linguistic input alone, yet humans appear to ground their understanding of words in sensorimotor experience. A natural solution is to augment LM representations with human judgments of a word's sensorimotor associations (e.g., the Lancaster Sensorimotor Norms), but this raises another challenge: most words are ambiguous, and judgments of words in isolation fail to account for this multiplicity of meaning (e.g., "wooden table" vs. "data table"). We attempted to address this problem by building a new lexical resource of contextualized sensorimotor judgments for 112 English words, each rated in four different contexts (448 sentences total). We show that these ratings encode overlapping but distinct information from the Lancaster Sensorimotor Norms, and that they also predict other measures of interest (e.g., relatedness), above and beyond measures derived from BERT. Beyond shedding light on theoretical questions, we suggest that these ratings could be of use as a "challenge set" for researchers building grounded language models.

show abstract

“…A fuller account of the constellation of factors involved in construal should also take seriously the grounded, situated nature of language use (Harnad, 1990;Kiros et al, 2018;Bender and Koller, 2020;Bisk et al, 2020). Frameworks motivated by the linguistic insights mentioned in §2 (such as the work on computational construction grammar referenced in §4) and by growing evidence of embodied simulations as the basis for meaning (Narayanan, 1999;Bergen and Chang, 2005;Feldman, 2006;Bergen, 2012;Tamari et al, 2020) are especially relevant lines of inquiry.…”

Section: Discussionmentioning

confidence: 99%

(Re)construing Meaning in NLP

Trott¹,

Torrent²,

Chang³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Human speakers have an extensive toolkit of ways to express themselves. In this paper, we engage with an idea largely absent from discussions of meaning in natural language understanding-namely, that the way something is expressed reflects different ways of conceptualizing or construing the information being conveyed. We first define this phenomenon more precisely, drawing on considerable prior work in theoretical cognitive semantics and psycholinguistics. We then survey some dimensions of construed meaning and show how insights from construal could inform theoretical and practical work in NLP.

show abstract

Language (Re)modelling: Towards Embodied Language Understanding

Cited by 11 publications

References 59 publications

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

A Metaverse: Taxonomy, Components, Applications, and Open Challenges

Contextualized Sensorimotor Norms: multi-dimensional measures of sensorimotor strength for ambiguous English words, in context

(Re)construing Meaning in NLP

Contact Info

Product

Resources

About