Abstract planning for reactive robots

Joshi, Saket; Schermerhorn, Paul; Khardon, Roni; Scheutz, Matthias

doi:10.1109/icra.2012.6225191

Cited by 9 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That is, assuming 𝑠𝑡𝑎𝑡𝑖𝑐 (𝑜) = ∅, σ𝑖−1 guarantees that ⟨𝑜 𝑖 , • • • , 𝑜 𝑛 ⟩ is a plan to σ𝑔 , while allowing the corresponding executor to terminate in the largest possible set of fluent states. Each such partial fluent state σ𝑠𝑔 is chosen as a subgoal, for which a new learner ℓ σ𝑠𝑔 is spawned and added to 𝐿 (lines [12][13][14][15][16][17][18]. σ𝑠𝑔 is also added to the set of "plannable" fluent states Σ 𝑝𝑙𝑎𝑛 .…”

Section: Learning Operator Policiesmentioning

confidence: 99%

SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning

Sarathy¹,

Kasenberg²,

Goel³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Symbolic planning models allow decision-making agents to sequence actions in arbitrary ways to achieve a variety of goals in dynamic domains. However, they are typically handcrafted and tend to require precise formulations that are not robust to human error. Reinforcement learning (RL) approaches do not require such models, and instead learn domain dynamics by exploring the environment and collecting rewards. However, RL approaches tend to require millions of episodes of experience and often learn policies that are not easily transferable to other tasks. In this paper, we address one aspect of the open problem of integrating these approaches: how can decision-making agents resolve discrepancies in their symbolic planning models while attempting to accomplish goals? We propose an integrated framework named SPOTTER that uses RL to augment and support ("spot") a planning agent by discovering new operators needed by the agent to accomplish goals that are initially unreachable for the agent. SPOTTER outperforms pure-RL approaches while also discovering transferable symbolic knowledge and does not require supervision, successful plan traces or any a priori knowledge about the missing planning operator.

show abstract

Section: Learning Operator Policiesmentioning

confidence: 99%

SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning

Sarathy¹,

Kasenberg²,

Goel³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Case notation and FOADDs have been used to implement approximate linear programming [18,51] and approximate policy iteration via linear programming [52] and FODDs have been used to implement relational policy iteration [53]. GFODDs have also been used for open world reasoning and applied in a robotic context [54]. The work of [55] and [56] explore SDP solutions, with GFODDs and case notation respectively, to relational partially observable MDPs (POMDPs) where the problem is conceptually and algorithmically much more complex.…”

Section: Deductive Lifted Stochastic Planningmentioning

confidence: 99%

Stochastic Planning and Lifted Inference

Khardon¹,

Sanner²

2017

Preprint

Self Cite

View full text Add to dashboard Cite

Lifted probabilistic inference (Poole, 2003) and symbolic dynamic programming for lifted stochastic planning (Boutilier et al, 2001) were introduced around the same time as algorithmic efforts to use abstraction in stochastic systems. Over the years, these ideas evolved into two distinct lines of research, each supported by a rich literature. Lifted probabilistic inference focused on efficient arithmetic operations on template-based graphical models under a finite domain assumption while symbolic dynamic programming focused on supporting sequential decisionmaking in rich quantified logical action models and on open domain reasoning. Given their common motivation but different focal points, both lines of research have yielded highly complementary innovations. In this chapter, we aim to help close the gap between these two research areas by providing an overview of lifted stochastic planning from the perspective of probabilistic inference, showing strong connections to other chapters in this book. This also allows us to define generalized lifted inference as a paradigm that unifies these areas and elucidates open problems for future research that can benefit both lifted inference and stochastic planning.

show abstract

“…GFODDs were introduced together with a set of operations that can be used to manipulate and combine functions and in this way provide a tool for computation with numerical functions over possible worlds. Prior work includes implementation of the FODD fragment where the only aggregation operator allowed is max [17,15] and more recently implementations for GFODDs with max and average aggregations [19,18]. In this paper we investigate several computational questions for GFODDs with min and max aggregation.…”

Section: Fodds and Gfodds And Their Computational Problemsmentioning

confidence: 99%

The complexity of reasoning with FODD and GFODD

Hescott

Khardon

2015

Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Recent work introduced Generalized First Order Decision Diagrams (GFODD) as a knowledge representation that is useful in mechanizing decision theoretic planning in relational domains. GFODDs generalize function-free first order logic and include numerical values and numerical generalizations of existential and universal quantification. Previous work presented heuristic inference algorithms for GFODDs and implemented these heuristics in systems for decision theoretic planning. In this paper, we study the complexity of the computational problems addressed by such implementations. In particular, we study the evaluation problem, the satisfiability problem, and the equivalence problem for GFODDs under the assumption that the size of the intended model is given with the problem, a restriction that guarantees decidability. Our results provide a complete characterization placing these problems within the polynomial hierarchy. The same characterization applies to the corresponding restriction of problems in first order logic, giving an interesting new avenue for efficient inference when the number of objects is bounded. Our results show that for Σ k formulas, and for corresponding GFODDs, evaluation and satisfiability are Σ p k complete, and equivalence is Π p k+1 complete. For Π k formulas evaluation is Π p k complete, satisfiability is one level higher and is Σ p k+1 complete, and equivalence is Π p k+1 complete.

show abstract

Abstract planning for reactive robots

Cited by 9 publications

References 15 publications

SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning

SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning

Stochastic Planning and Lifted Inference

The complexity of reasoning with FODD and GFODD

Contact Info

Product

Resources

About