Physical Causality of Action Verbs in Grounded Language Understanding

Gao, Qiaozi; Doering, Malcolm; Yang, Shaolong; Chai, Joyce

doi:10.18653/v1/p16-1171

Cited by 26 publications

(22 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semantic Role Labeling (SRL) follows a sequential tagging approach, implemented through Conditional Random Field (CRF). The problem is further stressed in [16], where Gao and colleagues studied a specific sub-category of the action verbs, namely the result verbs, that are meant to cause a change of state in the patient referred by the verb itself. In their framework, given a video and a caption, the aim is to ground different semantic roles of the verb to objects in the video, relying on the physical causality of verbs (i.e., physical changes that a verb may arouse within the environment) as features in a CRF model.…”

Section: Related Workmentioning

confidence: 99%

Grounded language interpretation of robotic commands through structured learning

Vanzo

Croce

Bastianelli

et al. 2020

Artificial Intelligence

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Grounded language interpretation of robotic commands through structured learning

Vanzo

Croce

Bastianelli

et al. 2020

Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…Although commonsense inference between action verbs and result verbs has been described in linguistic studies (Rappaport Hovav and Levin, 2010), there is still a lack of detailed account of potential causality that could be denoted by an action verb (Gao et al, 2016).…”

Section: Related Workmentioning

confidence: 99%

“…For instance, in the largest domain-independent computational verb lexicon VerbNet (Kipper Schuler, 2005), that provides semantic role representation for 6394 verbs (version 3.2b), the action verb hit and the result verb break have the same structure: [Agent, Instrument, Patient, Result]. Even if the semantic representation for a verb may indicate that a change of state is involved, it does not provide the specifics associated with the verb's meaning (e.g., to what attribute of its patient the changes might occur) (Gao et al, 2016).…”

Section: Related Workmentioning

confidence: 99%

“…To make such transformations possible, previous works (Misra et al, 2015;She and Chai, 2016) explicitly model verbs with predicates describing the resulting states of actions. Their empirical evaluations have demonstrated how incorporating result states into verb representations can link language with underlying planning modules for robotic systems (Gao et al, 2016). Recent investigations use reinforcement learning to transform language commands into primitive actions (Misra et al, 2017) or representation of actions (Arumugam et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Commonsense inference in human-robot communication

Huminski¹,

Ng²,

Kwok³

et al. 2019

Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing

View full text Add to dashboard Cite

Natural language communication between machines and humans are still constrained. The article addresses a gap in natural language understanding about actions, specifically that of understanding commands. We propose a new method for commonsense inference (grounding) of high-level natural language commands into specific action commands for further execution by a robotic system. The method allows to build a knowledge base that consists of a large set of commonsense inferences. The preliminary results have been presented.

show abstract

“…The grounding of language relations by visual data only starts to emerge. For instance, [26] ground causality of action verbs by using video data. We know very little on how we could use visual and multimodal representations to improve the accuracy and efficiency of language understanding by machines.…”

Section: Current Proposed Solutions: Representation Learning and Deepmentioning

confidence: 99%

Argumentation mining: How can a machine acquire common sense and world knowledge?

Moens

2018

AAC

View full text Add to dashboard Cite

Abstract. Argumentation mining is an advanced form of human language understanding by the machine. This is a challenging task for a machine. When sufficient explicit discourse markers are present in the language utterances, the argumentation can be interpreted by the machine with an acceptable degree of accuracy. However, in many real settings, the mining task is difficult due to the lack or ambiguity of the discourse markers, and the fact that a substantial amount of knowledge needed for the correct recognition of the argumentation, its composing elements and their relationships is not explicitly present in the text, but makes up the background knowledge that humans possess when interpreting language. In this article 1 we focus on how the machine can automatically acquire the needed common sense and world knowledge. As very few research has been done in this respect, many of the ideas proposed in this article are tentative, but start being researched.We give an overview of the latest methods for human language understanding that map language to a formal knowledge representation that facilitates other tasks (for instance, a representation that is used to visualize the argumentation or that is easily shared in a decision or argumentation support system). Most current systems are trained on texts that are manually annotated. Then we go deeper into the new field of representation learning that nowadays is very much studied in computational linguistics. This field investigates methods for representing language as statistical concepts or as vectors, allowing straightforward methods of compositionality. The methods often use deep learning and its underlying neural network technologies to learn concepts from large text collections in an unsupervised way (i.e., without the need for manual annotations). We show how these methods can help the argumentation mining process, but also demonstrate that these methods need further research to automatically acquire the necessary background knowledge and more specifically common sense and world knowledge. We propose a number of ways to improve the learning of common sense and world knowledge by exploiting textual and visual data, and touch upon how we can integrate the learned knowledge in the argumentation mining process.

show abstract

Physical Causality of Action Verbs in Grounded Language Understanding

Cited by 26 publications

References 30 publications

Grounded language interpretation of robotic commands through structured learning

Grounded language interpretation of robotic commands through structured learning

Commonsense inference in human-robot communication

Argumentation mining: How can a machine acquire common sense and world knowledge?

Contact Info

Product

Resources

About