Pathfinding in stochastic environments: learning vs planning

Abstract

Among the main challenges associated with navigating a mobile robot in complex environments are partial observability and stochasticity. This work proposes a stochastic formulation of the pathfinding problem, assuming that obstacles of arbitrary shapes may appear and disappear at random moments of time. Moreover, we consider the case when the environment is only partially observable for an agent. We study and evaluate two orthogonal approaches to tackle the problem of reaching the goal under such conditions: planning and learning. Within planning, an agent constantly re-plans and updates the path based on the history of the observations using a search-based planner. Within learning, an agent asynchronously learns to optimize a policy function using recurrent neural networks (we propose an original efficient, scalable approach). We carry on an extensive empirical evaluation of both approaches that show that the learning-based approach scales better to the increasing number of the unpredictably appearing/disappearing obstacles. At the same time, the planning-based one is preferable when the environment is close-to-the-deterministic (i.e., external disturbances are rare). Code available at https://github.com/Tviskaron/pathfinding-in-stochastic-envs.

A Comprehensive Review on Leveraging Machine Learning for Multi-Agent Path Finding

“…• Flatland [4] specifically addresses train pathfinding challenges, including situations involving train malfunctions. It achieves a performance of 156 FPS with 80 agents [165].…”

“…• POGEMA [165] is purpose-built for addressing partially-observable MAPF challenges. It showcases an impressive capability of 83,000 FPS in scenarios with 80 agents.…”

“…Citation information: DOI 10.1109/ACCESS.2024.3392305This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ (a) MiniGrid[164] (b) MiniWorld[164] (c) FlatLand[4] (d) POGEMA[165] (e) RWARE[166] (f) VMAS[167]…”

Reinforcement Learning with Success Induced Task Prioritization

“…The hyperparameters were tuned ones on procedurally generated maps without curriculum learning. We use the same network architecture as in [15,16].…”