Everything robots always wanted to know about housework (but were afraid to ask)

Abstract

In this paper we discuss the problem of actionspecific knowledge processing, representation and acquisition by autonomous robots performing everyday activities. We report on a thorough analysis of the household domain which has been performed on a large corpus of natural-language instructions from the Web, which underlines the supreme need of action-specific knowledge for robots acting in those environments. We introduce the concept of Probabilistic Robot Action Cores (PRAC) that are well-suited for encoding such knowledge in a probabilistic first-order knowledge base. We additionally show how such a knowledge base can be acquired by natural language and we address the problems of incompleteness, underspecification and ambiguity of naturalistic action specifications and point out how PRAC models can tackle those.

Multi-purpose natural language understanding linked to sensorimotor experience in humanoid robots

“…In [24], NL instructions are processed with a dependency parser and background axioms are used to make assumptions and fill the gaps in the NL input. In [25], background knowledge about robot actions is axiomatized using Markov Logic Networks. In [26], a knowledge base of known actions, objects, and locations is used for a Bayesbased grounding model.…”

“…Symbolic approaches work well for small pre-defined domains, but most of them employ manually written rules, which limits their coverage and scalability. In order to increase the linguistic coverage, some of the systems use lexical-semantic resources like WordNet, FrameNet, and VerbNet [27], [25]. In this study, we follow this approach and generate our lexical axioms from Wordnet and FrameNet.…”

“…The framework presented in this paper is in line with this approach, because abductive inference considers both the linguistic and perceptual input as an observation to be interpreted given the background knowledge. b) Planning: With respect to the action execution, the existing approaches can be classified into those directly mapping NL instructions into action sequences [33], [34], [25] and those employing a planner [35], [27], [24], [36]. We employ a planner, because it allows us to account for the dynamically changing environment, which is essential for the human-robot collaboration.…”

Manipulation action tree bank: A knowledge resource for humanoids

“…The logical extension of these works is to make tree banks of the actions themselves and use them to reason about actions in a language like manner. Also, in [21] a probabilistic first-order knowledge base was built as action-specific knowledge for robots acting in household environments. In this paper we further focus on manipulation actions, and propose to build tree banks for manipulation actions to serve as a hierarchical knowledge base for autonomous humanoids to effectively perform reasoning and prediction over the learned semantic structures.…”

Learning haptic representation for manipulating deformable food objects

“…Some robotics work focused on food manipulation such as [14], [15] that reason about how objects are changed by actions through a rule-based ontology and build a knowledge base from natural language. Bollini et al [16] presented a vision based cookie baking system that uses compliant controllers similar to ours.…”