Hindsight Optimization for Probabilistic Planning with Factored Actions

Issakkimuthu, Murugeswari; Fern, Alan; Khardon, Roni; Tadepalli, Prasad; Xue, Shan

doi:10.1609/icaps.v25i1.13711

Cited by 4 publications

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Current state-of-theart planners for RDDL problems are based on online search, where at each step some combination of search and reasoning is used to select an action. For example, there are planners based on sample-based tree search (Keller and Eyerich 2012;Kolobov et al 2012;Bonet and Geffner 2012), symbolic variants (Cui et al 2015;Raghavan et al 2015;Anand et al 2016), and those that construct and solve integer linear programs at each step (Issakkimuthu et al 2015). These planners can require non-trivial computation time per step, which can make them inapplicable to problems that require fast decisions.…”

Section: Introductionmentioning

confidence: 99%

Training Deep Reactive Policies for Probabilistic Planning Problems

Issakkimuthu

Fern

Tadepalli

2018

ICAPS

Self Cite

View full text Add to dashboard Cite

State-of-the-art probabilistic planners typically apply look- ahead search and reasoning at each step to make a decision. While this approach can enable high-quality decisions, it can be computationally expensive for problems that require fast decision making. In this paper, we investigate the potential for deep learning to replace search by fast reactive policies. We focus on supervised learning of deep reactive policies for probabilistic planning problems described in RDDL. A key challenge is to explore the large design space of network architectures and training methods, which was critical to prior deep learning successes. We investigate a number of choices in this space and conduct experiments across a set of benchmark problems. Our results show that effective deep reactive policies can be learned for many benchmark problems and that leveraging the planning problem description to define the network structure can be beneficial.

show abstract

Section: Introductionmentioning

confidence: 99%

Training Deep Reactive Policies for Probabilistic Planning Problems

Issakkimuthu

Fern

Tadepalli

2018

ICAPS

Self Cite

View full text Add to dashboard Cite

show abstract

Leveraging Multiple Environments for Learning and Decision Making: a Dismantling Use Case

Suárez-Hernández

Gaugry

Segovia-Aguas

et al. 2020

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Learning is usually performed by observing real robot executions. Physics-based simulators are a good alternative for providing highly valuable information while avoiding costly and potentially destructive robot executions. We present a novel approach for learning the probabilities of symbolic robot action outcomes. This is done leveraging different environments, such as physics-based simulators, in execution time. To this end, we propose MENID (Multiple Environment Noise Indeterministic Deictic) rules, a novel representation able to cope with the inherent uncertainties present in robotic tasks. MENID rules explicitly represent each possible outcomes of an action, keep memory of the source of the experience, and maintain the probability of success of each outcome. We also introduce an algorithm to distribute actions among environments, based on previous experiences and expected gain. Before using physicsbased simulations, we propose a methodology for evaluating different simulation settings and determining the least timeconsuming model that could be used while still producing coherent results. We demonstrate the validity of the approach in a dismantling use case, using a simulation with reduced quality as simulated system, and a simulation with full resolution where we add noise to the trajectories and some physical parameters as a representation of the real system.

show abstract

Hindsight Optimization for Hybrid State and Action MDPs

Raghavan

Sanner

Khardon

et al. 2017

AAAI

View full text Add to dashboard Cite

Hybrid (mixed discrete and continuous) state and action Markov Decision Processes (HSA-MDPs) provide an expressive formalism for modeling stochastic and concurrent sequential decision-making problems. Existing solvers for HSA-MDPs are either limited to very restricted transition distributions, require knowledge of domain-specific basis functions to achieve good approximations, or do not scale. We explore a domain-independent approach based on the framework of hindsight optimization (HOP) for HSA-MDPs, which uses an upper bound on the finite-horizon action values for action selection. Our main contribution is a linear time reduction to a Mixed Integer Linear Program (MILP) that encodes the HOP objective, when the dynamics are specified as location-scale probability distributions parametrized by Piecewise Linear (PWL) functions of states and actions. In addition, we show how to use the same machinery to select actions based on a lower-bound generated by straight line plans. Our empirical results show that the HSA-HOP approach effectively scales to high-dimensional problems and outperforms baselines that are capable of scaling to such large hybrid MDPs.

show abstract

Hindsight Optimization for Probabilistic Planning with Factored Actions

Cited by 4 publications

References 16 publications

Training Deep Reactive Policies for Probabilistic Planning Problems

Training Deep Reactive Policies for Probabilistic Planning Problems

Leveraging Multiple Environments for Learning and Decision Making: a Dismantling Use Case

Hindsight Optimization for Hybrid State and Action MDPs

Contact Info

Product

Resources

About