Physics-Driven Machine Learning for Time-Optimal Path Planning in Stochastic Dynamic Flows

Chowdhury, Rohit; Subramani, Deepak N.

doi:10.1007/978-3-030-61725-7_34

Cited by 5 publications

(8 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Partially Observable Markov Decision Processes (POMDPs) have also been used for a variety of simultaneous localization and planning applications of ground or aerial vehicles where the focus is local closed-loop control [31], [32]. Building on top of the MDP framework, reinforcement learning (RL) algorithms [33], [34], [35], [36], [37], [38], [39] have also been used for path planning. GPU-accelerated RL libraries have also been developed recently [40], [41].…”

Section: A Previous Progress In Optimal Path Planningmentioning

confidence: 99%

“…Similar to the setup in [39] and [30], we discretize the domain to a spatio-temporal grid world that constitutes the state space (Fig 1(B)). A state s represents a square region in the spatio-temporal grid, which is indexed by a spatio-temporal coordinate [x s , t s ].…”

Section: A Mdp Formulation Of Path Planningmentioning

confidence: 99%

“…Note that here we emphasize a 2D in space problem setup; however, our framework and implementation apply to 3D in space as well. Also, we use a kinematic framework for planning, without considering the dynamics of the flow around the UAV [23], [28], [29], [30], [39].…”

Section: A Mdp Formulation Of Path Planningmentioning

confidence: 99%

“…Here, Einstein's notation (repeated indices indicate summation) is used. In the present paper, we employ the computationally efficient Dynamically Orthogonal field equations [53], in which the DO mean, modes and coefficients are solved by integrating explicit equations for them [26], [39]. Alternatively, any modeling system that provides the flow field's distribution in terms of the above decomposition is sufficient to compute the transition probability matrix and complete the planning.…”

Section: Stochastic Dynamic Environment Modelingmentioning

confidence: 99%

See 3 more Smart Citations

Optimal Path Planning of Autonomous Marine Vehicles in Stochastic Dynamic Ocean Flows using a GPU-Accelerated Algorithm

Chowdhury,

Subramani

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous marine vehicles play an essential role in many ocean science and engineering applications. Planning time and energy optimal paths for these vehicles to navigate in stochastic dynamic ocean environments is essential to reduce operational costs. In some missions, they must also harvest solar, wind, or wave energy (modeled as a stochastic scalar field) and move in optimal paths that minimize net energy consumption. Markov Decision Processes (MDPs) provide a natural framework for sequential decision making for robotic agents in such environments. However, building a realistic model and solving the modeled MDP becomes computationally expensive in large-scale real-time applications, warranting the need of parallel algorithms and efficient implementation. In the present work, we introduce an efficient end-to-end GPU-accelerated algorithm that (i) builds the MDP model (computing transition probabilities and expected one-step rewards); and (ii) solves the MDP to compute an optimal policy. We develop methodical and algorithmic solutions to overcome the limited global memory of GPUs by (i) using a dynamic reduced-order representation of the ocean flows, (ii) leveraging the sparse nature of the state transition probability matrix, (iii) introducing a neighbouring sub-grid concept and (iv) proving that it is sufficient to use only the stochastic scalar field's mean to compute the expected one-step rewards for missions involving energy harvesting from the environment; thereby saving memory and reducing the computational effort. We demonstrate the algorithm on a simulated stochastic dynamic environment and highlight that it builds the MDP model and computes the optimal policy 600-1000x faster than conventional CPU implementations, making it suitable for real-time use.

show abstract

Section: A Previous Progress In Optimal Path Planningmentioning

confidence: 99%

Section: A Mdp Formulation Of Path Planningmentioning

confidence: 99%

Section: A Mdp Formulation Of Path Planningmentioning

confidence: 99%

Section: Stochastic Dynamic Environment Modelingmentioning

confidence: 99%

See 2 more Smart Citations

Optimal Path Planning of Autonomous Marine Vehicles in Stochastic Dynamic Ocean Flows using a GPU-Accelerated Algorithm

Chowdhury,

Subramani

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The challenge is primarily the computational cost (e.g., [7][8][9]). Existing Markov Decision Process (MDP) [4] or Reinforcement Learning [10,11] path planners are slow for realistic real-time applications. Path planners based on Dijkstra's algorithm [12], variants of A * [13,14] and Delayed D * [15] work well in deterministic settings, but their Monte Carlo versions are computationally inefficient.…”

Section: Introductionmentioning

confidence: 99%

GPU-Accelerated Multi-Objective Optimal Planning in Stochastic Dynamic Environments

Chowdhury

Navsalkar

Subramani

2022

JMSE

View full text Add to dashboard Cite

The importance of autonomous marine vehicles is increasing in a wide range of ocean science and engineering applications. Multi-objective optimization, where trade-offs between multiple conflicting objectives are achieved (such as minimizing expected mission time, energy consumption, and environmental energy harvesting), is crucial for planning optimal routes in stochastic dynamic ocean environments. We develop a multi-objective path planner in stochastic dynamic flows by further developing and improving our recently developed end-to-end GPU-accelerated single-objective Markov Decision Process path planner. MDPs with scalarized rewards for multiple objectives are formulated and solved in idealized stochastic dynamic ocean environments with dynamic obstacles. Three simulated mission scenarios are completed to elucidate our approach and capabilities: (i) an agent moving from a start to target by minimizing travel time and net-energy consumption when harvesting solar energy in an uncertain flow; (ii) an agent moving from a start to target by minimizing travel time and-energy consumption with uncertainties in obstacle initial positions; (iii) an agent attempting to cross a shipping channel while avoiding multiple fast moving ships in an uncertain flow. Optimal operating curves are computed in a fraction of the time that would be required for existing solvers and algorithms. Crucially, our solution can serve as the benchmark for other approximate AI algorithms such as Reinforcement Learning and help improve explainability of those models.

show abstract

Navigating Underactuated Agents by Hitchhiking Forecast Flows

Wiggert

Doshi

Lermusiaux

et al. 2022

2022 IEEE 61st Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

In dynamic flow fields such as winds and ocean currents an agent can navigate by going with the flow, only using minimal propulsion to nudge itself into beneficial flows. This navigation paradigm of hitchhiking flows is highly energyefficient. However, reliable navigation in this setting remains challenging as typically only forecasts are available which differ significantly from the true currents and the forecast error can be larger than can be handled by the actuation of the agent. In this paper, we propose a novel control method for reliable navigation of underactuated agents hitchhiking flows based on imperfect forecasts. In the spirit of Model Predictive Control our method allows for time-optimal replanning at every time step with only one computation per forecast. Using the recent Multi-Time Hamilton-Jacobi Reachability formulation we obtain a value function which is then used for closed-loop control. We evaluate the reliability of our method empirically over a large set of multi-day start-target missions in the ocean currents of the Gulf of Mexico with realistic forecast errors. Our method outperforms the baselines significantly, achieving high reliability, measured as the success rate of navigating from start to target, at low computational cost.

show abstract

Physics-Driven Machine Learning for Time-Optimal Path Planning in Stochastic Dynamic Flows

Cited by 5 publications

References 15 publications

Optimal Path Planning of Autonomous Marine Vehicles in Stochastic Dynamic Ocean Flows using a GPU-Accelerated Algorithm

Optimal Path Planning of Autonomous Marine Vehicles in Stochastic Dynamic Ocean Flows using a GPU-Accelerated Algorithm

GPU-Accelerated Multi-Objective Optimal Planning in Stochastic Dynamic Environments

Navigating Underactuated Agents by Hitchhiking Forecast Flows

Contact Info

Product

Resources

About