Real-Time Path Planning using a Simulation-Based Markov Decision Process

Abstract

This paper introduces a novel path planning technique called MCRT which is aimed at non-deterministic, partially known, real-time domains populated with dynamically moving obstacles, such as might be found in a real-time strategy (RTS) game. The technique combines an efficient form of Monte-Carlo tree search with the randomized exploration capabilities of rapidly exploring random tree (RRT) planning. The main innovation of MCRT is in incrementally building an RRT structure with a collision-sensitive reward function, and then re-using it to efficiently solve multiple, sequential goals. We have implemented the technique in MCRT-planner, a program which solves non-deterministic path planning problems in imperfect information RTS games, and evaluated it in comparison to four other state of the art techniques. Planners embedding each technique were applied to a typical RTS game and evaluated using the game score and the planning cost. The empirical evidence demonstrates the success of MCRT-planner.

A Monte-Carlo path planner for dynamic and partially observable environments

“…The MCRT planner is a promising planner for path planning problems over multiple journeys and has been shown to perform better than LSS-LRTA in some previous studies [10]. Our results show that MOCART-CGA performs significantly better than the MCRT planner in the single journey path planning problems.…”

“…We present empirical results that demonstrate MOCART-CGA has stronger performance than a state-of-the-art MonteCarlo path planner [10], both in time to search for the next action and in plan quality. We also demonstrate that MOCART-CGA takes a smaller amount of time to search than current state-of-the-art path planning algorithms [11], [12].…”

“…The R function (Figure 4, adapted from [10]) computes the immediate reward of a state-action pair. At first, R estimates the result of applying an action a in a state s n by using the stochastic function T ransition(s n , a) (Line 1) (s next = T ransition(s n , a)).…”

“…The MCRT planner [10] is a Monte-Carlo Planner based on Rapidly-exploring Random Trees (RRT) [23], applied to an RTS game. MCRT performs better than LSS-LRTA for solving path planning problems in a typical RTS game when collecting resources.…”

A Self-Adaptive Context-Aware Model for Mobile Commerce

“…We use a variation of decision tree for the predication preferences, however, induction tree is applied on a summarized form of the problem space-a synopsis structure is used to represent the summary form of the problem space. We use a real-time Monte-Carlo sampling [3] to generate the synopsis structure. The main contributions of this work as follow:…”