Bellman goes relational

Kersting, Kristian; Otterlo, Martijn van; Raedt, Luc De

doi:10.1145/1015330.1015401

Cited by 63 publications

(53 citation statements)

References 12 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The idea is to construct minimal logical partitions of the state space required to make all necessary value function distinctions. For example, Kersting et al [13] present an exact value iteration for relational MDPs. Sanner et al [17] exploit factored transition models of first-order MPDs to approximate the value function based on linear combinations of abstract first-order value functions.…”

Section: Related Workmentioning

confidence: 99%

Relevance Grounding for Planning in Relational Domains

Lang

Toussaint

2009

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. Probabilistic relational models are an efficient way to learn and represent the dynamics in realistic environments consisting of many objects. Autonomous intelligent agents that ground this representation for all objects need to plan in exponentially large state spaces and large sets of stochastic actions. A key insight for computational efficiency is that successful planning typically involves only a small subset of relevant objects. In this paper, we introduce a probabilistic model to represent planning with subsets of objects and provide a definition of object relevance. Our definition is sufficient to prove consistency between repeated planning in partially grounded models restricted to relevant objects and planning in the fully grounded model. We propose an algorithm that exploits object relevance to plan efficiently in complex domains. Empirical results in a simulated 3D blocksworld with an articulated manipulator and realistic physics prove the effectiveness of our approach.

show abstract

Section: Related Workmentioning

confidence: 99%

Relevance Grounding for Planning in Relational Domains

Lang

Toussaint

2009

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

show abstract

“…The fourth contrast is with those methods [13,14,19,24,29] that rely upon learning, that is, upon training agents to perform well in simulated environments. Here the evolving experience of the agent is effectively translated into merit-oriented weightings of the alternative actions available to each perception.…”

Section: Positioningmentioning

confidence: 99%

Designing Effective Policies for Minimal Agents

Broda

Hogger

2008

The Computer Journal

View full text Add to dashboard Cite

Abstract. A policy for a minimal reactive agent is a set of condition-action rules used to determine its response to perceived environmental stimuli. When the policy pre-disposes the agent to achieving a stipulated goal we call it a teleo-reactive policy. This paper presents a framework for constructing and evaluating teleo-reactive policies for one or more minimal agents, based upon discounted-reward evaluation of policy-restricted subgraphs of complete situation-graphs. The main feature of the method is that it exploits explicit and definite associations of the agent's perceptions with states. The combinatorial burden that would potentially ensue from such associations can be ameliorated by suitable use of abstractions. The framework allows one to plan for a number of agents by focusing upon the behaviour of a single representative of them. It allows for varied behaviour to be modelled, including communication between agents. Simulation results presented here indicate that the method affords a good degree of scalability and predictive power.

show abstract

“…The key observation is that each RMDP induces a traditional MDP [15], which can be obtained by starting in some initial ground state and then applying each abstract transition until no more new ground states can be computed. Thus, the existence of an optimal policy π for each (resulting) ground MDP is guaranteed.…”

Section: Relational Navigation Policiesmentioning

confidence: 99%

“…Later, Dietterich and Flann [22] combined this idea with reinforcement learning by associating these generalized state descriptions with values obtained from value iteration. Subsequently, Boutilier et al [23] and Kersting et al [15] generalized Dietterich and Flann's approach to relational domains, i.e., RMDPs. Recently, Mausam and Weld [10] suggested to approximate the value function by inducing a relational regression tree from observed traces.…”

Section: Relational Navigation Policiesmentioning

confidence: 99%

Learning Relational Navigation Policies

Cocora

Kersting

Plagemann

et al. 2006

2006 IEEE/RSJ International Conference on Intelligent Robots and Systems

Self Cite

View full text Add to dashboard Cite

Abstract-Navigation is one of the fundamental tasks for a mobile robot. The majority of path planning approaches has been designed to entirely solve the given problem from scratch given the current and goal configurations of the robot. Although these approaches yield highly efficient plans, the computed policies typically do not transfer to other, similar tasks. We propose to learn relational decision trees as abstract navigation strategies from example paths. Relational abstraction has several interesting and important properties. First, it allows a mobile robot to generalize navigation plans from specific examples provided by users or exploration. Second, the navigation policy learned in one environment can be transferred to unknown environments. In several experiments with real robots in a real environment and in simulated runs, we demonstrate the usefulness of our approach.

show abstract

Bellman goes relational

Cited by 63 publications

References 12 publications

Relevance Grounding for Planning in Relational Domains

Relevance Grounding for Planning in Relational Domains

Designing Effective Policies for Minimal Agents

Learning Relational Navigation Policies

Contact Info

Product

Resources

About