Integrating Answer Set Programming with Semantic Dictionaries for Robot Task Planning

Lu, Dongcai; Zhou, Yi; Wu, Feng; Zhang, Zhao; Chen, Xiaoping

doi:10.24963/ijcai.2017/609

Cited by 17 publications

(13 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Executing natural language instruction given to a robot is a well-studied problem, particularly for object fetching and navigational instruction. However, the existing works in the literature mostly focus on instruction understanding for plan generation [3,4] and assume that a generated plan can be executed without failure or further human intervention. Natural language instructions are prone to ambiguity and incompleteness that are often tackled using dialogue [5] and knowledge-based reasoning [6].…”

Section: Related Workmentioning

confidence: 99%

Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot

Pramanick,

Sarkar,

Banerjee

et al. 2021

Preprint

View full text Add to dashboard Cite

The utility of collocating robots largely depends on the easy and intuitive interaction mechanism with the human. If a robot accepts task instruction in natural language, first, it has to understand the user's intention by decoding the instruction. However, while executing the task, the robot may face unforeseeable circumstances due to the variations in the observed scene and therefore requires further user intervention. In this article, we present a system called Talk-to-Resolve (TTR) that enables a robot to initiate a coherent dialogue exchange with the instructor by observing the scene visually to resolve the impasse. Through dialogue, it either finds a cue to move forward in the original plan, an acceptable alternative of the original plan, or affirmation to abort the task altogether. To realize the possible stalemate, we utilize the dense captions of the observed scene and the given instruction jointly to compute the robot's next action. We evaluate our system based on a data set of initial instruction and situational scene pairs. Our system can identify the stalemate and resolve them with appropriate dialogue exchange with 82% accuracy. Additionally, a user study reveals that the questions from our systems are more natural (4.02 on average on a scale of 1 to 5) as compared to a state-of-the-art (3.08 on average).

show abstract

Section: Related Workmentioning

confidence: 99%

Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot

Pramanick,

Sarkar,

Banerjee

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This lessens the effort to re-use our system in a new domain. Though Lu et al [25] and Munawar et al [26] proposed approaches to convert the parsed semantic information to a formal language for planners, we extend this by a complete framework that uses human-robot dialogues and context-sensitive task planning, leading to highly accurate conversion of natural language instructions to planned sequences of primitive actions. Moreover, we also tackle the problem of handling compound instructions containing multiple tasks.…”

Section: Related Workmentioning

confidence: 99%

Enabling Human-Like Task Identification From Natural Conversation

Pramanick

Sarkar

Balamuralidhar

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

A robot as a coworker or a cohabitant is becoming mainstream day-by-day with the development of low-cost sophisticated hardware. However, an accompanying software stack that can aid the usability of the robotic hardware remains the bottleneck of the process, especially if the robot is not dedicated to a single job. Programming a multipurpose robot requires an on the fly mission scheduling capability that involves task identification and plan generation. The problem dimension increases if the robot accepts tasks from a human in natural language. Though recent advances in NLP and planner development can solve a variety of complex problems, their amalgamation for a dynamic robotic task handler is used in a limited scope. Specifically, the problem of formulating a planning problem from natural language instructions is not studied in details. In this work, we provide a non-trivial method to combine an NLP engine and a planner such that a robot can successfully identify tasks and all the relevant parameters and generate an accurate plan for the task. Additionally, some mechanism is required to resolve the ambiguity or missing pieces of information in natural language instruction. Thus, we also develop a dialogue strategy that aims to gather additional information with minimal question-answer iterations and only when it is necessary. This work makes a significant stride towards enabling a human-like task understanding capability in a robot.

show abstract

“…More precisely, user instructions or high-level tasks are parsed into a series of frame elements based on predefined knowledge. These frame templates come from open resources [14] or domain experts [15], for example, the words ''drink'' and ''fridge'' in the user task ''Serve a drink from fridge'' can be filled into the elements ''Theme'' and ''Source'' of the frame ''Bringing,'' respectively. Although this task has been extensively studied, intent understanding is still not considered to be a problem that is solved.…”

Section: Related Workmentioning

confidence: 99%

“…Especially, for SRTR, the results of SF, ID, and sentence-level semantic frame-filling are improved by 1.7%, 1.1%, and 1.7%, respectively.INDEX TERMS Human-robot interaction, service robots, slot-gated mechanism, spoken language understanding, verb context feature. ambiguity of verbs and incomplete natural language description scenarios [14]. In recent years, with the development of recurrent neural networks, some joint training models based on Long Short-Term Memory(LSTM) and its variants have become widely used in natural language understanding.…”

mentioning

confidence: 99%

A Novel Slot-Gated Model Combined With a Key Verb Context Feature for Task Request Understanding by Service Robots

Zhang

Jiang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Spoken language understanding (SLU) is a fundamental to service robot handling of natural language task requests. There are two main basic problems in SLU, namely, intent determination (ID) and slot filling (SF). The slot-gated recurrent neural network joint model for the two tasks has been proven to be superior to the single model, and has achieved the most advanced performance. However, in the context of task requests for home service robots, there exists a phenomenon that the information about a current word is strongly dependent on key verbs in the sentence, and it is difficult to capture this relation well with current methods. In this paper, we extract the key instructional verb containing greater core task information based on dependency parsing, and construct a feature that combines the key verb with its contextual information to solve this problem. To further improve the performance of the slot-gated model, we consider the strong relations between intent and slot. By introducing intent attention vectors into the slot attention vectors through the global-level gate and element-level gate, a novel dual slot-gated mechanism is proposed to explicitly model the complex relations between the results of the ID tasks and SF prediction tasks and optimize the global prediction results. Our experimental results on the ATIS dataset and an extended home service task (SRTR) dataset based on FrameNet show that the proposed method outperforms the most advanced methods in both tasks. Especially, for SRTR, the results of SF, ID, and sentence-level semantic frame-filling are improved by 1.7%, 1.1%, and 1.7%, respectively.INDEX TERMS Human-robot interaction, service robots, slot-gated mechanism, spoken language understanding, verb context feature. ambiguity of verbs and incomplete natural language description scenarios [14]. In recent years, with the development of recurrent neural networks, some joint training models based on Long Short-Term Memory(LSTM) and its variants have become widely used in natural language understanding. These models are different from traditional verb matching and other context-free grammar models. They can learn the remote dependencies in sentences through the gating mechanism, and improve the overall performance through joint training of ID and SF. However, this kind of model only adds training loss for the joint training, which cannot model the complex correlation between the two tasks well. Moreover, in the field of service robots, people usually interact with robots in a command language. Unlike auxiliary query languages, many commands in this field contain key verbs that are directly related to behavioral intent. There is a strong information complement relation between many chunks in the command and such key verbs, while there is a weak dependence relation with other chunks. In the example of ''After five minutes, please wipe the table with a paper towel.'' shown in Figure 1 (a), the information of ''Time'', ''Tool'' and ''Target Object'' of ''wipe'' are complemented by ''After five minutes...

show abstract

Integrating Answer Set Programming with Semantic Dictionaries for Robot Task Planning

Cited by 17 publications

References 16 publications

Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot

Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot

Enabling Human-Like Task Identification From Natural Conversation

A Novel Slot-Gated Model Combined With a Key Verb Context Feature for Task Request Understanding by Service Robots

Contact Info

Product

Resources

About