Deep Reinforcement Learning Applied to Multi-agent Informative Path Planning in Environmental Missions

Luis, Samuel Yanes; Esteve, Manuel Perales; Reina, Daniel Gutiérrez; Toral, S. L.

doi:10.1007/978-3-031-26564-8_2

Cited by 4 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One may choose to use RL in such systems as it allows agents to learn coordinated policies that may be complicated and time-consuming to hard code. Yanes Luis et al (2022) uses DQN with prioritised experience replay to develop a multi-agent framework for IPP. The proposed method is tested and applied to the Peralta et al (2020) .…”

Section: Methodologies In Active Environmental Monitoringmentioning

confidence: 99%

A survey on autonomous environmental monitoring approaches: towards unifying active sensing and reinforcement learning

Mansfield,

Montazeri

2024

Front. Robot. AI

View full text Add to dashboard Cite

The environmental pollution caused by various sources has escalated the climate crisis making the need to establish reliable, intelligent, and persistent environmental monitoring solutions more crucial than ever. Mobile sensing systems are a popular platform due to their cost-effectiveness and adaptability. However, in practice, operation environments demand highly intelligent and robust systems that can cope with an environment’s changing dynamics. To achieve this reinforcement learning has become a popular tool as it facilitates the training of intelligent and robust sensing agents that can handle unknown and extreme conditions. In this paper, a framework that formulates active sensing as a reinforcement learning problem is proposed. This framework allows unification with multiple essential environmental monitoring tasks and algorithms such as coverage, patrolling, source seeking, exploration and search and rescue. The unified framework represents a step towards bridging the divide between theoretical advancements in reinforcement learning and real-world applications in environmental monitoring. A critical review of the literature in this field is carried out and it is found that despite the potential of reinforcement learning for environmental active sensing applications there is still a lack of practical implementation and most work remains in the simulation phase. It is also noted that despite the consensus that, multi-agent systems are crucial to fully realize the potential of active sensing there is a lack of research in this area.

show abstract

Section: Methodologies In Active Environmental Monitoringmentioning

confidence: 99%

A survey on autonomous environmental monitoring approaches: towards unifying active sensing and reinforcement learning

Mansfield,

Montazeri

2024

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…Additionally, the partial observability of our proposed environment is mitigated by implementing a first phase of homogeneous patrolling, as the pollution distribution along the lake at the beginning of the episode is unknown. In the context of multi-agent patrolling of environmental missions, [6] and [28] proposed a scheme that employs Deep Q-Learning with a Convolutional Neural Network as a shared fleet policy and utilizes a global visual state. In [6], a decoupled final layer was proposed for N agents with |A| possible actions.…”

Section: Related Workmentioning

confidence: 99%

“…In fully cooperative environments that employ joint reward signals, agents face credit assignment challenges when determining the impact of their actions on team performance. To tackle the credit assignment problem, [28] proposed a decoupled reward in which each agent receives a reward only for their individual contributions, without any additional considerations. They demonstrated that their approach is effective in addressing the credit assignment problem, which motivates this work to employ it.…”

Section: Related Workmentioning

confidence: 99%

Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach

Diop,

Luis,

Esteve

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This study proposes the use of an Autonomous Surface Vehicle (ASV) fleet with water quality sensors for efficient patrolling to monitor water resource pollution. This is formulated as a Patrolling Problem, which consists of planning and executing efficient routes to continuously monitor a given area. When patrolling Lake Ypacaraí with ASVs, the scenario transforms into a Partially Observable Markov Game (POMG) due to unknown pollution levels. Given the computational complexity, a Multi-Agent Deep Reinforcement Learning (MADRL) approach is adopted, with a common policy for homogeneous agents. A consensus algorithm assists in collision avoidance and coordination. The work introduces exploration and reinforcement phases to the patrolling problem. The Exploration Phase aims at homogeneous map coverage, while the Intensification Phase prioritizes high polluted areas. The innovative introduction of a transition variable, ν, efficiently controls the transition from exploration to intensification. Results demonstrate the superiority of the method, which outperforms a Single-Phase (trained on a single task) Deep Q-Network (DQN) by an average of 17% on the intensification task. The proposed multitask learning approach with parameter sharing, coupled with DQN training, outperforms Task-Specific DQN (two DQNs trained on separate tasks) by 6% in exploration and 13% in intensification. It also outperforms the heuristic-based Lawn Mower Path Planner (LMPP) and Random Wanderer Path Planner (RWPP) algorithms, by 35% and 20% on average respectively. Additionally, it outperforms a Particle Swarm Optimization-based Path Planner (PSOPP) by an average of 26%. The algorithm demonstrates adaptability in unforeseen scenarios, giving users flexibility in configuration.

show abstract

Learning-based methods for adaptive informative path planning

Popović,

Ott,

Rückin

et al. 2024

Robotics and Autonomous Systems

View full text Add to dashboard Cite

Deep Reinforcement Learning Applied to Multi-agent Informative Path Planning in Environmental Missions

Cited by 4 publications

References 26 publications

A survey on autonomous environmental monitoring approaches: towards unifying active sensing and reinforcement learning

A survey on autonomous environmental monitoring approaches: towards unifying active sensing and reinforcement learning

Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach

Learning-based methods for adaptive informative path planning

Contact Info

Product

Resources

About