PALMER: Perception-Action Loop with Memory for Long-Horizon Planning

Beker, Onur; Mohammadi, Mohammad Reza; Zamir, Amir

doi:10.48550/arxiv.2212.04581

Cited by 1 publication

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluate our method on two complex point maze environments and a novel image-based ViZDoom environment which have been used as a benchmark in RL navigation tasks (Zhang et al 2020;Nachum et al 2018;Beker, Mohammadi, and Zamir 2022). These maps include obstacles (impenetrable) and hazards (high cost but penetrable).…”

Section: Methodsmentioning

confidence: 99%

“…Why distributional RL? For rewards, we need to estimate just the expected V π (s, s ′ ), but it is known from the literature that learning the distribution of V π and then calculating expected value leads to better estimates (Eysenbach, Salakhutdinov, and Levine 2019;Beker, Mohammadi, and Zamir 2022). For completeness, we provide experimental evidence of this phenomenon in Appendix.…”

Section: Approachmentioning

confidence: 99%

“…We present a thorough comparison of our ConstrainedRRT* to SORB's planner in Section . PALMER (Beker, Mohammadi, and Zamir 2022) employs RRT* for the high level, but instead of distributional RL at the low-level it uses an offline RL like approach, requiring a large pre-collected dataset fully covering the environment; importantly, PALMER also cannot enforce constraints.…”

Section: Contributionsmentioning

confidence: 99%

See 2 more Smart Citations

Handling Long and Richly Constrained Tasks through Constrained Hierarchical Reinforcement Learning

Lu,

Sinha,

Varakantham

2024

AAAI

View full text Add to dashboard Cite

Safety in goal directed Reinforcement Learning (RL) settings has typically been handled through constraints over trajectories and have demonstrated good performance in primarily short horizon tasks. In this paper, we are specifically interested in the problem of solving temporally extended decision making problems such as robots cleaning different areas in a house while avoiding slippery and unsafe areas (e.g., stairs) and retaining enough charge to move to a charging dock; in the presence of complex safety constraints. Our key contribution is a (safety) Constrained Search with Hierarchical Reinforcement Learning (CoSHRL) mechanism that combines an upper level constrained search agent (which computes a reward maximizing policy from a given start to a far away goal state while satisfying cost constraints) with a low-level goal conditioned RL agent (which estimates cost and reward values to move between nearby states). A major advantage of CoSHRL is that it can handle constraints on the cost value distribution (e.g., on Conditional Value at Risk, CVaR) and can adjust to flexible constraint thresholds without retraining. We perform extensive experiments with different types of safety constraints to demonstrate the utility of our approach over leading approaches in constrained and hierarchical RL.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Approachmentioning

confidence: 99%

Section: Contributionsmentioning

confidence: 99%

See 1 more Smart Citation