Action learning and grounding in simulated human–robot interactions

Abstract

In order to enable robots to interact with humans in a natural way, they need to be able to autonomously learn new tasks. The most natural way for humans to tell another agent, which can be a human or robot, to perform a task is via natural language. Thus, natural human–robot interactions also require robots to understand natural language, i.e. extract the meaning of words and phrases. To do this, words and phrases need to be linked to their corresponding percepts through grounding. Afterward, agents can learn the optimal micro-action patterns to reach the goal states of the desired tasks. Most previous studies investigated only learning of actions or grounding of words, but not both. Additionally, they often used only a small set of tasks as well as very short and unnaturally simplified utterances. In this paper, we introduce a framework that uses reinforcement learning to learn actions for several tasks and cross-situational learning to ground actions, object shapes and colors, and prepositions. The proposed framework is evaluated through a simulated interaction experiment between a human tutor and a robot. The results show that the employed framework can be used for both action learning and grounding.

Special issue on adaptive and learning agents 2018

“…The paper Action learning and grounding in simulated human-robot interactions by Roesler and Nowé (2019) deals with the problem of developing robots that can understand and act on commands given in human language. Their framework is set up to learn the meaning of object, action, colour and preposition words and phrases.…”

Interactive Explanations: Diagnosis and Repair of Reinforcement Learning Based Agent Behaviors

“…One way to reduce the time of experts is using an inverse RL algorithm [29] to find a base policy and reward function for the problem at hand. To facilitate the generation of explanations with our framework, we could implement object grounding techniques [30] so non-experts in RL could teach the bot the meaning of objects and actions using natural language.…”

Second Grand-Challenge and Workshop on Multimodal Language (Challenge-HML)

“…through the use of probabilistic models , to ground words through percepts in artificial agents. This section describes an online CSL algorithm for grounding of words, which has first been proposed by Roesler and Nowé (2018) and recently been extended with auxiliary word and phrase detection (Roesler and Nowé, 2019). Since the sentences in this study are shorter, have a much simpler structure, and less variation than the sentences used in (Roesler and Nowé, 2019), the previous auxiliary word and phrase detection algorithms do not work.…”

“…coca cola or lemonade instead of bottle. In this paper, a recently proposed unsupervised online grounding framework (Roesler and Nowé, 2019) is extended to handle real percepts obtained during human-robot interactions. More specifically, the learning framework is extended to first convert obtained percepts through clustering to an abstract representation, which is then used to ground all non-auxiliary words 1 of the encountered natural language instructions through cross-situational learning.…”

“…Both auxiliary word mechanisms, i.e. the one used in(Roesler and Nowé, 2019) and the one proposed in this study, are used in parallel because both have shown to not produce false detections, i.e. they either detect an auxiliary word correctly or do not detect it.…”