Enabling Human-Like Task Identification From Natural Conversation

Abstract

A robot as a coworker or a cohabitant is becoming mainstream day-by-day with the development of low-cost sophisticated hardware. However, an accompanying software stack that can aid the usability of the robotic hardware remains the bottleneck of the process, especially if the robot is not dedicated to a single job. Programming a multipurpose robot requires an on the fly mission scheduling capability that involves task identification and plan generation. The problem dimension increases if the robot accepts tasks from a human in natural language. Though recent advances in NLP and planner development can solve a variety of complex problems, their amalgamation for a dynamic robotic task handler is used in a limited scope. Specifically, the problem of formulating a planning problem from natural language instructions is not studied in details. In this work, we provide a non-trivial method to combine an NLP engine and a planner such that a robot can successfully identify tasks and all the relevant parameters and generate an accurate plan for the task. Additionally, some mechanism is required to resolve the ambiguity or missing pieces of information in natural language instruction. Thus, we also develop a dialogue strategy that aims to gather additional information with minimal question-answer iterations and only when it is necessary. This work makes a significant stride towards enabling a human-like task understanding capability in a robot.

Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot

“…Executing natural language instruction given to a robot is a well-studied problem, particularly for object fetching and navigational instruction. However, the existing works in the literature mostly focus on instruction understanding for plan generation [3,4] and assume that a generated plan can be executed without failure or further human intervention. Natural language instructions are prone to ambiguity and incompleteness that are often tackled using dialogue [5] and knowledge-based reasoning [6].…”

“…However, these systems only focus on the linguistic information provided by the human and do not take the uncertainty of the robot's perception into account when attempting to execute a plan. For example, in our earlier works, we have handled the natural language task instruction parsing to generate a high-level execution plan for the robot [3,7]. These systems are supported by a dialogue engine that can raise a suitable query for the human user if the robot could not understand the task [8].…”

“…The task is decomposed into a sequence of robot executable actions by task planning. To generate such a plan from the instruction, we utilize Conditional Random Field (CRF) models introduced in our previous work [3,7]. The CRF models are trained to recognize a set of generic tasks with the corresponding arguments from text input, as shown in Table 2.…”

“…Given the task type and arguments mentioned in the instruction, we generate a high-level task plan to execute the task. We encode the instruction in a PDDL planning problem [24] by using templates of initial and goal conditions for the predicted task and use a forward-search planner to generate the plan [3]. An example of a task plan is given below.…”

DeComplex: Task planning from complex natural instructions by a collocating robot

“…Problem description. Existing works that translate natural language instruction to a sequence of actions, either constrain the instruction to a single task [5], [6] or assume multiple tasks are performed sequentially [7], [8], [9]. In the later case, the sequence is assumed to be the order in which the tasks appear in the instruction.…”