Emergence of divisional cooperation with negotiation and re-learning and evaluation of flexibility in continuous cooperative patrol problem

Sugiyama, Ayumi; Sea, Vourchteang; Sugawara, Toshiharu

doi:10.1007/s10115-018-1285-8

Cited by 8 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sea et al [7] improved the traditional k-means clustering algorithm based on non-uniform visiting frequency and workload to achieve graph partition and subgraph patrolling. Sugiyama et al [13] proposed a negotiation-based learning method to address the multi-robots patrol problem. Wiandt et al [14] proposed a self-organized partition algorithm to achieve distributed partition, in which robots transmit information by broadcasting to other robots without a third-party coordinator.…”

Section: Related Workmentioning

confidence: 99%

“…In every patrol, the constraint expressed by Equations (11) and (12) ensure that the USV starts from the mother ship before patrolling and return to the mother ship after completing a patrol. The constraint in Equation (13) guarantees that the number of arrivals and departures at a target are the same. Moreover, Equation (14) guarantees that all the targets are visited with the required number of visits.…”

Section: Problem Formulationmentioning

confidence: 99%

See 1 more Smart Citation

Hybrid Partition-Based Patrolling Scheme for Maritime Area Patrol with Multiple Cooperative Unmanned Surface Vehicles

Xie

Zhou

Luo

et al. 2020

JMSE

View full text Add to dashboard Cite

Multi-robot cooperative patrolling systems have been extensively employed in the civilian and military fields, including monitoring forest fires, marine search-and-rescue, and area patrol. Multi-robot area patrol problems refer to the activity that a team of robots works cooperatively and regularly to visit the key targets in the given area for security. Following consideration of the low cost and high safety of unmanned surface vehicles (USV), a team of USVs is organized to perform area patrol in a sophisticated maritime environment. In this paper, we establish a mathematical model considering the characteristics of the cooperative patrol task and the limited conditions of USVs. A hybrid partition-based patrolling scheme is proposed for a multi-USV system to visit targets with different importance levels in a maritime area. Firstly, a centralized area partition algorithm is utilized to partition the patrolling area according to the number of USVs. Secondly, a distributed path planning algorithm is applied to planning the patrolling path for each USV to visit the targets in a maritime environment to minimize the length of the patrolling path for the USV team. Finally, comparative experiments between the proposed scheme and other methods are carried out to validate the performance of the hybrid partition-based patrolling scheme. Simulation results and experimental analysis show the efficiency of the proposed hybrid partition-based patrolling scheme compared to several previous patrolling algorithms.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Problem Formulationmentioning

confidence: 99%

Hybrid Partition-Based Patrolling Scheme for Maritime Area Patrol with Multiple Cooperative Unmanned Surface Vehicles

Xie

Zhou

Luo

et al. 2020

JMSE

View full text Add to dashboard Cite

show abstract

“…In recent years, the exploration of a dynamic environment has been carried out in many applications, such as robot navigation in a dynamic environment [6] and a patrol task in a dynamic environment [7]. Transfer learning is a common method for solutions in those works, owning the advantage of timesaving for multi-agent learning to adapt to environmental changes [8], by using negotiation and knowledge transfer methods to cope with environment changes [9,10], by combining the concept of incremental learning with the dynamic environment in the evolutionary strategies [11], by adopting a scalable transfer learning framework to solve dynamic environments [12], by using neural networks to transmit information on changes in the environment [13], and by using the deep convolutional transfer learning model (DCTL) to cope with changing tasks [14].…”

Section: Introductionmentioning

confidence: 99%

A Multi-Agent Adaptive Co-Evolution Method in Dynamic Environments

Zhang

et al. 2023

Mathematics

View full text Add to dashboard Cite

It is challenging to ensure satisfying co-evolution efficiency for the multi-agents in dynamic environments since during Actor-Critic training there is a high probability of falling into local optimality, failing to adapt to the suddenly changed environment quickly. To solve this problem, this paper proposes a multi-agent adaptive co-evolution method in dynamic environments (ACE-D) based on the classical multi-agent reinforcement learning method MADDPG, which effectively realizes self-adaptive new environments and co-evolution in dynamic environments. First, an experience screening policy is introduced based on the MADDPG method to reduce the negative influence of original environment experience on exploring new environments. Then, an adaptive weighting policy is applied to the policy network, which accordingly generates benchmarks for varying environments and assigns higher weights to those policies that are more beneficial for new environments exploration, so that to save time while promoting adaptability of the agents. Finally, different types of dynamic environments with complexity at different levels are built to verify the co-evolutionary effects of the two policies separately and the ACE-D method comprehensively. The experimental results demonstrate that, compared with a range of other methods, the ACE-D method has obvious advantages helping multi-agent adapt to dynamic environments and preventing them from falling into local optima, with more than 25% improvement in stable reward and more than 23% improvement in training efficiency. The ACE-D method is valuable and commendable to promote the co-evolutionary effect of multi-agent in dynamic environments.

show abstract

“…When these agents are expected to be used for complex tasks or in large environments, multiple agents are required to complete these tasks by coordinating and cooperating with each other to achieve their own goals or shared goals. There is a wide range of multi-agent system applications, for example, traffic flow control [1], robots in an automated warehouse [15], cooperative security surveillance [13], and airplane operation control [7]. However, appropriate coordinated behavior is sophisticated, and just taking optimal actions based on agents' independent decisions may lead to conflicts with other agents, such as competition for shared and limited resources and physical collisions.…”

Section: Introductionmentioning

confidence: 99%

Deadlock-Free Method for Multi-Agent Pickup and Delivery Problem Using Priority Inheritance with Temporary Priority

Fujitani¹,

Yamauchi²,

Miyashita³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a control method for the multi-agent pickup and delivery problem (MAPD problem) by extending the priority inheritance with backtracking (PIBT) method to make it applicable to more general environments. PIBT is an effective algorithm that introduces a priority to each agent, and at each timestep, the agents, in descending order of priority, decide their next neighboring locations in the next timestep through communications only with the local agents. Unfortunately, PIBT is only applicable to environments that are modeled as a bi-connected area, and if it contains dead-ends, such as tree-shaped paths, PIBT may cause deadlocks. However, in the real-world environment, there are many dead-end paths to locations such as the shelves where materials are stored as well as loading/unloading locations to transportation trucks. Our proposed method enables MAPD tasks to be performed in environments with some tree-shaped paths without deadlock while preserving the PIBT feature; it does this by allowing the agents to have temporary priorities and restricting agents' movements in the trees. First, we demonstrate that agents can always reach their delivery without deadlock. Our experiments indicate that the proposed method is very efficient, even in environments where PIBT is not applicable, by comparing them with those obtained using the well-known token passing method as a baseline.

show abstract

Emergence of divisional cooperation with negotiation and re-learning and evaluation of flexibility in continuous cooperative patrol problem

Cited by 8 publications

References 13 publications

Hybrid Partition-Based Patrolling Scheme for Maritime Area Patrol with Multiple Cooperative Unmanned Surface Vehicles

Hybrid Partition-Based Patrolling Scheme for Maritime Area Patrol with Multiple Cooperative Unmanned Surface Vehicles

A Multi-Agent Adaptive Co-Evolution Method in Dynamic Environments

Deadlock-Free Method for Multi-Agent Pickup and Delivery Problem Using Priority Inheritance with Temporary Priority

Contact Info

Product

Resources

About