Censored deep reinforcement patrolling with information criterion for monitoring large water resources using Autonomous Surface Vehicles

Luis, Samuel Yanes; Reina, Daniel Gutiérrez; Toral, S. L.

doi:10.1016/j.asoc.2022.109874

Cited by 3 publications

(3 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As mentioned above, the Censoring-DQL [5] algorithm is employed to ensure deterministic computation that can address actions leading to nonnavigable zones. In this algorithm, invalid actions of an agent i that would lead to nonnavigable zones are identified using the information in the lake map, which is known a priori.…”

Section: Collision Avoidance Mechanismsmentioning

confidence: 99%

“…Manual sampling is impractical due to the lake's vast size (60 km 2 ) and the potential health risks for biologists in contaminated environments. Furthermore, installing a fixed sensor grid is not an optimal solution because it limits the ability to vary sampling locations and may be more costly due to frequent battery replacements compared to human-conducted missions [5].…”

Section: Introductionmentioning

confidence: 99%

“…As a result, each individual reward can be modeled with a single network. Concerning the nonnavigable terrain constraints, Censoring-DQL [5] is applied to takes the deterministic information of the environment to neglect the actions that would violate those constraints. Furthermore, if a possible collision between agents is about to occur, a consensus algorithm proposed in [9] is used to avoid it.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach

Diop,

Luis,

Esteve

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This study proposes the use of an Autonomous Surface Vehicle (ASV) fleet with water quality sensors for efficient patrolling to monitor water resource pollution. This is formulated as a Patrolling Problem, which consists of planning and executing efficient routes to continuously monitor a given area. When patrolling Lake Ypacaraí with ASVs, the scenario transforms into a Partially Observable Markov Game (POMG) due to unknown pollution levels. Given the computational complexity, a Multi-Agent Deep Reinforcement Learning (MADRL) approach is adopted, with a common policy for homogeneous agents. A consensus algorithm assists in collision avoidance and coordination. The work introduces exploration and reinforcement phases to the patrolling problem. The Exploration Phase aims at homogeneous map coverage, while the Intensification Phase prioritizes high polluted areas. The innovative introduction of a transition variable, ν, efficiently controls the transition from exploration to intensification. Results demonstrate the superiority of the method, which outperforms a Single-Phase (trained on a single task) Deep Q-Network (DQN) by an average of 17% on the intensification task. The proposed multitask learning approach with parameter sharing, coupled with DQN training, outperforms Task-Specific DQN (two DQNs trained on separate tasks) by 6% in exploration and 13% in intensification. It also outperforms the heuristic-based Lawn Mower Path Planner (LMPP) and Random Wanderer Path Planner (RWPP) algorithms, by 35% and 20% on average respectively. Additionally, it outperforms a Particle Swarm Optimization-based Path Planner (PSOPP) by an average of 26%. The algorithm demonstrates adaptability in unforeseen scenarios, giving users flexibility in configuration.

show abstract

Section: Collision Avoidance Mechanismsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach

Diop,

Luis,

Esteve

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

Deep Reinforcement Multiagent Learning Framework for Information Gathering with Local Gaussian Processes for Water Monitoring

Yanes Luis,

Shutin,

Marchal Gómez

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

The conservation of hydrological resources involves continuously monitoring their contamination. A multiagent system composed of autonomous surface vehicles is proposed herein to efficiently monitor the water quality. To achieve a safe control of the fleet, the fleet policy should be able to act based on measurements and fleet state. It is proposed to use local Gaussian processes and deep reinforcement learning to jointly obtain effective monitoring policies. Local Gaussian processes, unlike classical global Gaussian processes, can accurately model the information in a dissimilar spatial correlation which captures more accurately the water quality information. A deep convolutional policy is proposed, that bases the decisions on the observation on the mean and variance of this model, by means of an information gain reward. Using a double deep Q‐learning algorithm, agents are trained to minimize the estimation error in a safe manner thanks to a Consensus‐based heuristic. Simulation results indicate an improvement of up to 24% in terms of the mean absolute error with the proposed models. Also, training results with 1–3 agents indicate that our proposed approach returns 20% and 24% smaller average estimation errors for, respectively, monitoring water quality variables and monitoring algae blooms, as compared to state‐of‐the‐art approaches.

show abstract

Deep Variational Auto-Encoder for Model-Based Water Quality Patrolling with Intelligent Surface Vehicles

Yanes Luis,

Basilico,

Antonazzi

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Censored deep reinforcement patrolling with information criterion for monitoring large water resources using Autonomous Surface Vehicles

Cited by 3 publications

References 38 publications

Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach

Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach

Deep Reinforcement Multiagent Learning Framework for Information Gathering with Local Gaussian Processes for Water Monitoring

Deep Variational Auto-Encoder for Model-Based Water Quality Patrolling with Intelligent Surface Vehicles

Contact Info

Product

Resources

About