BWIBots: A platform for bridging the gap between AI and human–robot interaction research

Khandelwal, Piyush; Zhang, Shiqi; Sinapov, Jivko; Leonetti, Matteo; Thomason, Jesse; Yang, Fangkai; Gori, Ilaria; Svetlik, Maxwell; Khante, Priyanka; Lifschitz, Vladimir; Aggarwal, J. K.; Mooney, Raymond J.; Stone, Peter

doi:10.1177/0278364916688949

Cited by 86 publications

(47 citation statements)

References 65 publications

(85 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ontology theory is also applied to other fields, such as industrial production collaboration [27] and navigation in indoor scenarios [28,29]. The work in References [27,30] portrays the domain knowledge for the robot task planning in logic language BC [31,32]. However, this approach is disadvantageous in sharing knowledge and updating the new knowledge inferred.…”

Section: Introductionmentioning

confidence: 99%

RTPO: A Domain Knowledge Base for Robot Task Planning

Sun

Chen

2019

Electronics

View full text Add to dashboard Cite

Knowledge can enhance the intelligence of robots’ high-level decision-making. However, there is no specific domain knowledge base for robot task planning in this field. Aiming to represent the knowledge in robot task planning, the Robot Task Planning Ontology (RTPO) is first designed and implemented in this work, so that robots can understand and know how to carry out task planning to reach the goal state. In this paper, the RTPO is divided into three parts: task ontology, environment ontology, and robot ontology, followed by a detailed description of these three types of knowledge, respectively. The OWL (Web Ontology Language) is adopted to represent the knowledge in robot task planning. Then, the paper proposes a method to evaluate the scalability and responsiveness of RTPO. Finally, the corresponding task planning algorithm is designed based on RTPO, and then the paper conducts experiments on the basis of the real robot TurtleBot3 to verify the usability of RTPO. The experimental results demonstrate that RTPO has good performance in scalability and responsiveness, and the robot can achieve given high-level tasks based on RTPO.

show abstract

Section: Introductionmentioning

confidence: 99%

RTPO: A Domain Knowledge Base for Robot Task Planning

Sun

Chen

2019

Electronics

View full text Add to dashboard Cite

show abstract

“…However, these robots either work in human-forbidden environments, or have no interaction with humans except for obstacle avoidance. Researchers are developing mobile robot platforms that are able to interact with people in everyday, human-inhabited environments [1]- [4]. Some of the robot platforms can learn from the experience of human-robot interaction (HRI) to improve their language skills, e.g., learning new synonyms [5], but none of them learn entirely new entities.…”

Section: Introductionmentioning

confidence: 99%

“…Although such goaloriented dialog systems have been implemented on robots, few of them can learn to improve their language capabilities or augment their KB from the experience of human-robot conversations in the real world (details in Section II). 1 This work focuses on dialog-based robot knowledge augmentation, where the agent must identify when it is necessary to augment its KB and where in the KB to do that, as applied to our Segway-based mobile robot shown in Fig. 1.…”

Section: Introductionmentioning

confidence: 99%

Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog

Amiri

Bajracharya

Goktolgal

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Some robots can interact with humans using natural language, and identify service requests through humanrobot dialog. However, few robots are able to improve their language capabilities from this experience. In this paper, we develop a dialog agent for robots that is able to interpret user commands using a semantic parser, while asking clarification questions using a probabilistic dialog manager. This dialog agent is able to augment its knowledge base and improve its language capabilities by learning from dialog experiences, e.g., adding new entities and learning new ways of referring to existing entities. We have extensively evaluated our dialog system in simulation as well as with human participants through MTurk and real-robot platforms. We demonstrate that our dialog agent performs better in efficiency and accuracy in comparison to baseline learning agents. Demo video can be found at https://youtu.be/DFB3jbHBqYE

show abstract

“…Besides simple motion planning, possible applications include self-driving cars [13], autonomous trading agents in the stock market [42], or service robots [22].…”

Section: Introductionmentioning

confidence: 99%

Human-in-the-Loop Synthesis for Partially Observable Markov Decision Processes

Carr

Jansen

Wimmer

et al. 2018

2018 Annual American Control Conference (ACC)

View full text Add to dashboard Cite

We study planning problems where autonomous agents operate inside environments that are subject to uncertainties and not fully observable. Partially observable Markov decision processes (POMDPs) are a natural formal model to capture such problems. Because of the potentially huge or even infinite belief space in POMDPs, synthesis with safety guarantees is, in general, computationally intractable. We propose an approach that aims to circumvent this difficulty: in scenarios that can be partially or fully simulated in a virtual environment, we actively integrate a human user to control an agent. While the user repeatedly tries to safely guide the agent in the simulation, we collect data from the human input. Via behavior cloning, we translate the data into a strategy for the POMDP. The strategy resolves all nondeterminism and non-observability of the POMDP, resulting in a discrete-time Markov chain (MC). The efficient verification of this MC gives quantitative insights into the quality of the inferred human strategy by proving or disproving given system specifications. For the case that the quality of the strategy is not sufficient, we propose a refinement method using counterexamples presented to the human. Experiments show that by including humans into the POMDP verification loop we improve the state of the art by orders of magnitude in terms of scalability.

show abstract

BWIBots: A platform for bridging the gap between AI and human–robot interaction research

Cited by 86 publications

References 65 publications

RTPO: A Domain Knowledge Base for Robot Task Planning

RTPO: A Domain Knowledge Base for Robot Task Planning

Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog

Human-in-the-Loop Synthesis for Partially Observable Markov Decision Processes

Contact Info

Product

Resources

About