Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction

Abstract

As a new generation of cognitive robots start to enter our lives, it is important to enable robots to follow human commands and to learn new actions from human language instructions. To address this issue, this paper presents an approach that explicitly represents verb semantics through hypothesis spaces of fluents and automatically acquires these hypothesis spaces by interacting with humans. The learned hypothesis spaces can be used to automatically plan for lower-level primitive actions towards physical world interaction. Our empirical results have shown that the representation of a hypothesis space of fluents, combined with the learned hypothesis selection algorithm, outperforms a previous baseline. In addition, our approach applies incremental learning, which can contribute to life-long learning from humans in the future.

Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication

“…Specifically for verb semantics, recent works explored the connection between verbs and action planning Misra et al, 2014Misra et al, , 2015She and Chai, 2016), for example, by representing grounded verbs semantics as the desired goal state of the physical world that is a result of the corresponding actions. Such representations are learned based on example actions demonstrated by humans.…”

“…It is environment-independent and more generalizable. In (She and Chai, 2016), instead of one hypothesis, it maintains a specific-to-general hypothesis space as shown in Figure 2 to capture all goal hypotheses of a particular verb frame. Specifically, it assumes that one verb frame may lead to different outcomes under different environments, where each possible outcome is represented by one node in the hierarchical graph and each node is a conjunction of multiple atomic fluents.…”

“…Only demonstration (She and Chai, 2016) Interactive learning (this work) Execution Example Execution Example H1: Boil the water.…”

“…To support learning of grounded verb semantics, previous works Misra et al, 2015;She and Chai, 2016) rely on multiple instances of human demonstrations of corresponding actions. From these demonstrations, robots capture the state change of the environment caused by the actions and represent verb semantics as the desired goal state.…”

“…First of all, previous approaches were developed under the assumption of perfect perception of the environment Misra et al, 2015;She and Chai, 2016). However, this assumption does not hold in real-world situated interaction.…”