Extending persistent monitoring by combining ocean models and Markov Decision Processes

Al-Sabban, Wesam H.; Gonzalez, Felipé; Smith, Ryan N.

doi:10.1109/oceans.2012.6404931

Cited by 12 publications

(13 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All experiments are performed in a simulation environment that models the high-level dynamics of unmanned underwater vehicles (UUVs) operating in environments with stochastic ocean currents, available at https://github.com/fimdp/fimdpenv. The environment models the currents (flow velocity and heading) based on [17]. Each scenario consists of several agents navigating in twodimensional grid of cells.…”

Section: Numerical Examplesmentioning

confidence: 99%

See 1 more Smart Citation

Polynomial-Time Algorithms for Multi-Agent Minimal-Capacity Planning

Cubuktepe¹,

Blahoudek²,

Topcu³

2021

Preprint

View full text Add to dashboard Cite

We study the problem of minimizing the resource capacity of autonomous agents cooperating to achieve a shared task. More specifically, we consider high-level planning for a team of homogeneous agents that operate under resource constraints in stochastic environments and share a common goal: given a set of target locations, ensure that each location will be visited infinitely often by some agent almost surely. We formalize the dynamics of agents by consumption Markov decision processes. In a consumption Markov decision process, the agent has a resource of limited capacity. Each action of the agent may consume some amount of the resource. To avoid exhaustion, the agent can replenish its resource to full capacity in designated reload states. The resource capacity restricts the capabilities of the agent. The objective is to assign target locations to agents, and each agent is only responsible for visiting the assigned subset of target locations repeatedly. Moreover, the assignment must ensure that the agents can carry out their tasks with minimal resource capacity. We reduce the problem of finding target assignments for a team of agents with the lowest possible capacity to an equivalent graph-theoretical problem. We develop an algorithm that solves this graph problem in time that is polynomial in the number of agents, target locations, and size of the consumption Markov decision process. We demonstrate the applicability and scalability of the algorithm in a scenario where hundreds of unmanned underwater vehicles monitor hundreds of locations in environments with stochastic ocean currents.

show abstract

Section: Numerical Examplesmentioning

confidence: 99%

“…We demonstrate the applicability and the scalability of the algorithms on synthesizing optimal paths for persistent ocean monitoring using autonomous underwater vehicles [17]. The presented benchmark models the dynamics in the presence of stochastic ocean currents by consumption MDPs.…”

Section: Introductionmentioning

confidence: 99%

Polynomial-Time Algorithms for Multi-Agent Minimal-Capacity Planning

Cubuktepe¹,

Blahoudek²,

Topcu³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We demonstrate the utility of CMDPs and FIMDP on high-level planning tasks for unmanned underwater vehicles (UUVs) operating in ocean with stochastic currents. FIMD-PENV models this scenario based on [22]. The model discretizes the area of interest into a 2D grid-world.…”

Section: A Tools Examples and Evaluation Settingmentioning

confidence: 99%

Efficient Strategy Synthesis for MDPs with Resource Constraints

Blahoudek¹,

Novotný²,

Ornik³

et al. 2021

Preprint

View full text Add to dashboard Cite

We consider qualitative strategy synthesis for the formalism called consumption Markov decision processes. This formalism can model dynamics of an agents that operates under resource constraints in a stochastic environment. The presented algorithms work in time polynomial with respect to the representation of the model and they synthesize strategies ensuring that a given set of goal states will be reached (once or infinitely many times) with probability 1 without resource exhaustion. In particular, when the amount of resource becomes too low to safely continue in the mission, the strategy changes course of the agent towards one of a designated set of reload states where the agent replenishes the resource to full capacity; with sufficient amount of resource, the agent attempts to fulfill the mission again.We also present two heuristics that attempt to reduce expected time that the agent needs to fulfill the given mission, a parameter important in practical planning. The presented algorithms were implemented and numerical examples demonstrate (i) the effectiveness (in terms of computation time) of the planning approach based on consumption Markov decision processes and (ii) the positive impact of the two heuristics on planning in a realistic example.

show abstract

“…2. A glider is moving up and down by transitioning between two water current layers [18]. feedback plan π is defined as a function π : X → U which produces an action u = π(x) ∈ U (x), for any state x ∈ X, to reach the goal state x G .…”

Section: B Problem Formulationmentioning

confidence: 99%

Feedback Motion Planning for Long-Range Autonomous Underwater Vehicles

Orioke

Alam

Quinn

et al. 2019

OCEANS 2019 - Marseille

View full text Add to dashboard Cite

Ocean ecosystems have spatiotemporal variability and dynamic complexity that require a long-term deployment of an autonomous underwater vehicle for data collection. A new long-range autonomous underwater vehicle called Tethys is adapted to study different oceanic phenomena. Additionally, an ocean environment has external forces and moments along with changing water currents which are generally not considered in a vehicle kinematic model. In this scenario, it is not enough to generate a simple trajectory from an initial location to a goal location in an uncertain ocean as the vehicle can deviate from its intended trajectory. As such, we propose to compute a feedback plan that adapts the vehicle trajectory in the presence of any modeled or unmodeled uncertainties. In this work, we present a feedback motion planning method for the Tethys vehicle by combining a predictive ocean model and its kinematic modeling. Given a goal location, the Tethys kinematic model, and the water flow pattern, our method computes a feedback plan for the vehicle in a dynamic ocean environment that reduces its energy consumption. The computed feedback plan provides the optimal action for the Tethys vehicle to take from any location of the environment to reach the goal location considering its orientation. Our results based on actual ocean model prediction data demonstrate the applicability of our method.

show abstract

Extending persistent monitoring by combining ocean models and Markov Decision Processes

Cited by 12 publications

References 18 publications

Polynomial-Time Algorithms for Multi-Agent Minimal-Capacity Planning

Polynomial-Time Algorithms for Multi-Agent Minimal-Capacity Planning

Efficient Strategy Synthesis for MDPs with Resource Constraints

Feedback Motion Planning for Long-Range Autonomous Underwater Vehicles

Contact Info

Product

Resources

About