Modelling the Single Word to Multi-Word Transition Using Matrix Completion

Pizzuto, Gabriella; Hospedales, Timothy M.; Capirci, Olga; Cangelosi, Angelo

doi:10.1109/devlrn.2019.8850708

Cited by 1 publication

(1 citation statement)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using human descriptions of objects together with deictic hand gestures, researchers can train a grounding system for identifying referent objects (Perera & Allen, 2013;Matuszek, Bo, Zettlemoyer, & Fox, 2014;Whitney, Eldon, Oberlin, & Tellex, 2016;Whitney, Rosen, MacGlashan, Wong, & Tellex, 2017;Pizzuto, Hospedales, Capirci, & Cangelosi, 2019). Other researchers have focused on learning categorical properties of objects (red ) together with relational (taller ) and differentiating (differ by weight) properties of objects by exploring them with a robotic arm (Sinapov, Schenck, & Stoytchev, 2014b).…”

Section: Language Grounding With Human-robot Interactionmentioning

confidence: 99%

Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog

Thomason

Padmakumar²,

Sinapov³

et al. 2020

jair

109

View full text Add to dashboard Cite

In this work, we present methods for using human-robot dialog to improve language understanding for a mobile robot agent. The agent parses natural language to underlying semantic meanings and uses robotic sensors to create multi-modal models of perceptual concepts like red and heavy. The agent can be used for showing navigation routes, delivering objects to people, and relocating objects from one location to another. We use dialog clari_cation questions both to understand commands and to generate additional parsing training data. The agent employs opportunistic active learning to select questions about how words relate to objects, improving its understanding of perceptual concepts. We evaluated this agent on Amazon Mechanical Turk. After training on data induced from conversations, the agent reduced the number of dialog questions it asked while receiving higher usability ratings. Additionally, we demonstrated the agent on a robotic platform, where it learned new perceptual concepts on the y while completing a real-world task.

show abstract