Controlled Natural Languages for language generation in artificial cognition

Kirk, Nicholas H.; Nyga, Daniel; Beetz, Michael

doi:10.1109/icra.2014.6907843

Cited by 9 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The accuracy of the learnt ontology entries can be refined via reinforcement learning during the course of task executions, or be verified thanks to an active learning process. More precisely, the SVO entries can be encapsulated in questions in a process called language generation, in order to ask for their correctness [15]. d) Limitations: Although we assume the text is confined to a technical domain, the authors of the source might make use of partly figurative wordings.…”

Section: Discussionmentioning

confidence: 99%

Towards Learning Object Affordance Priors from Technical Texts

Kirk

2014

Preprint

Self Cite

View full text Add to dashboard Cite

Everyday activities performed by artificial assistants can potentially be executed naïvely and dangerously given their lack of common sense knowledge. This paper presents conceptual work towards obtaining prior knowledge on the usual modality (passive or active) of any given entity, and their affordance estimates, by extracting high-confidence ability modality semantic relations (X can Y relationship) from nonfigurative texts, by analyzing co-occurrence of grammatical instances of subjects and verbs, and verbs and objects. The discussion includes an outline of the concept, potential and limitations, and possible feature and learning framework adoption.

show abstract

Section: Discussionmentioning

confidence: 99%

Towards Learning Object Affordance Priors from Technical Texts

Kirk

2014

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Natural language is an ideal candidate, given that interfaces such as mouse-and-keyboard, touchscreens and programming languages are powerful, but require extensive training for proper usage [22]. Multiple facets of language-based human-robot interaction have been studied in literature, such as instruction understanding [23,24], motion plan generation [9,12,16,25], human-robot cooperation [26], semantic belief propagation [18,19], and visual language navigation [11,27]. Most of the recent works in the field have shifted from representing language in terms of classical grammatical structure towards data-driven techniques, due higher flexibility in knowledge representations [22].…”

Section: Related Workmentioning

confidence: 99%

Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers

Bucker¹,

Figueredo²,

Haddadin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Natural language is the most intuitive medium for us to interact with other people when expressing commands and instructions. However, using language is seldom an easy task when humans need to express their intent towards robots, since most of the current language interfaces require rigid templates with a static set of action targets and commands. In this work, we provide a flexible languagebased interface for human-robot collaboration, which allows a user to reshape existing trajectories for an autonomous agent. We take advantage of recent advancements in the field of large language models (BERT and CLIP) to encode the user command, and then combine these features with trajectory information using multi-modal attention transformers. We train the model using imitation learning over a dataset containing robot trajectories modified by language commands, and treat the trajectory generation process as a sequence prediction problem, analogously to how language generation architectures operate. We evaluate the system in multiple simulated trajectory scenarios, and show a significant performance increase of our model over baseline approaches. In addition, our realworld experiments with a robot arm show that users significantly prefer our natural language interface over traditional methods such as kinesthetic teaching or cost-function programming. Our study shows how the field of robotics can take advantage of large pre-trained language models towards creating more intuitive interfaces between robots and machines. Project webpage: https://arthurfenderbucker. github.io/NL_trajectory_reshaper/

show abstract

“…These instructions are grounded using WordNet (45) and CYC (69) and are captured as a set of instructions in a knowledge base. Later work (70) discussed controlled natural language as a way to repair missing information through explicit clarification. Nyga et al (71) used a similar probabilistic model to utilize relational knowledge to fill in gaps for aspects of the language missing from the workspace.…”

Section: Formal Reasoningmentioning

confidence: 99%

Robots That Use Language

Tellex

Gopalan

Kress-Gazit

et al. 2020

Annu. Rev. Control Robot. Auton. Syst.

135

View full text Add to dashboard Cite

This article surveys the use of natural language in robotics from a robotics point of view. To use human language, robots must map words to aspects of the physical world, mediated by the robot's sensors and actuators. This problem differs from other natural language processing domains due to the need to ground the language to noisy percepts and physical actions. Here, we describe central aspects of language use by robots, including understanding natural language requests, using language to drive learning about the physical world, and engaging in collaborative dialogue with a human partner. We describe common approaches, roughly divided into learning methods, logic-based methods, and methods that focus on questions of human–robot interaction. Finally, we describe several application domains for language-using robots.

show abstract

Controlled Natural Languages for language generation in artificial cognition

Cited by 9 publications

References 13 publications

Towards Learning Object Affordance Priors from Technical Texts

Towards Learning Object Affordance Priors from Technical Texts

Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers

Robots That Use Language

Contact Info

Product

Resources

About