Interactive task learning via embodied corrective feedback

Abstract

This paper addresses a task in Interactive Task Learning (Laird et al. IEEE Intell Syst 32:6–21, 2017). The agent must learn to build towers which are constrained by rules, and whenever the agent performs an action which violates a rule the teacher provides verbal corrective feedback: e.g. “No, red blocks should be on blue blocks”. The agent must learn to build rule compliant towers from these corrections and the context in which they were given. The agent is not only ignorant of the rules at the start of the learning process, but it also has a deficient domain model, which lacks the concepts in which the rules are expressed. Therefore an agent that takes advantage of the linguistic evidence must learn the denotations of neologisms and adapt its conceptualisation of the planning domain to incorporate those denotations. We show that by incorporating constraints on interpretation that are imposed by discourse coherence into the models for learning (Hobbs in On the coherence and structure of discourse, Stanford University, Stanford, 1985; Asher et al. in Logics of conversation, Cambridge University Press, Cambridge, 2003), an agent which utilizes linguistic evidence outperforms a strong baseline which does not.

Symbol Grounding and Task Learning from Imperfect Corrections

“…1 ∈ G but there are more red blocks than blue blocks in S * , making it impossible to place all of the red blocks. In such cases, the agent recovers by searching in the probabilistic neighbourhood of S * for alternatives from which a valid plan for achieving G can be constructed (Appelgren and Lascarides, 2020). The agent executes each action in its plan until it's completed or the teacher gives corrective feedback.…”

“…When the teacher designates a block o 3 on the table (thereby signaling that violation is indirect), the graphical model this generates is similar to Figure 4, save there are two additional nodes F o 3 and Red o 3 ∨ Blue o 3 (see (Appelgren and Lascarides, 2020) for details).…”

“…Our interest lies in natural language interactions where teachers can provide instructions (She et al, 2014), describe current states (Hristov et al, 2017) and define concepts (Scheutz et al, 2017), goals (Kirk and Laird, 2019), and actions (She et al, 2014), while the agent asks clarifying questions (She and Chai, 2017) and executes instructed commands. Teachers can also use corrective feedback (Appelgren and Lascarides, 2020). These approaches all assume that the teacher offers information that is both correct and timely.…”

“…In this work we focus on learning goals, which express constraints on a final state. The agent learns these goals by receiving corrective dialogue moves that highlight an aspect of the goal which the agent has violated (Appelgren and Lascarides, 2020).…”

Combining Cognitive Modeling and Reinforcement Learning for Clarification in Dialogue

“…Although we build on that work here, our prior work has not addressed the problem of learning context-sensitive clarification strategies. Appelgren and Lascarides (2020) also consider semantic grounding of color terms and exploit formal models of discourse coherence to do so; however, their work so far assumes dialogue policies (and primitive motor skills) to be fixed and known in advance.…”