Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog

Abstract

Some robots can interact with humans using natural language, and identify service requests through humanrobot dialog. However, few robots are able to improve their language capabilities from this experience. In this paper, we develop a dialog agent for robots that is able to interpret user commands using a semantic parser, while asking clarification questions using a probabilistic dialog manager. This dialog agent is able to augment its knowledge base and improve its language capabilities by learning from dialog experiences, e.g., adding new entities and learning new ways of referring to existing entities. We have extensively evaluated our dialog system in simulation as well as with human participants through MTurk and real-robot platforms. We demonstrate that our dialog agent performs better in efficiency and accuracy in comparison to baseline learning agents. Demo video can be found at https://youtu.be/DFB3jbHBqYE

Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog

“…We assume a closed world (A.5), but in general an embodied agent will encounter new people, objects, and parts of an environment. There are active lines of research regarding environment exploration (Wang et al, 2018), object discovery (Tucker, Aksaray, Paul, Stein, & Roy, 2017), and identifying missing referents via dialog (Amiri, Bajracharya, Goktolga, Thomason, & Zhang, 2019). These strategies are compatible with our current dialog framework, which grounds to and asks enumeration questions about all known, relevant referents via a visual depiction achievable by taking photos of new referents.…”

A survey of knowledge‐based sequential decision‐making under uncertainty

“…In a related approach, SDM has been used to manage human-robot dialog, which helps a robot acquire knowledge of synonyms (e.g., "java" and "coffee") that are used for RDK (Thomason et al 2015). Building on this work, other researchers have developed methods to add new object entities to the declarative knowledge in RDK-for-SDM systems (Amiri et al 2019). In other work, human (verbal) descriptions of observed robot behavior have been used to extract knowledge of previously unknown actions and action effects, which is merged with existing knowledge in the RDK component (Sridharan and Meadows 2018).…”

Learning to Ground Objects for Robot Task and Motion Planning

“…Task Planner: Our task planner P t is implemented using Answer Set Programming (ASP), which is a popular declarative language for knowledge representation and reasoning. ASP has been applied to task planning [8], [32], [36], [37].…”