Strategic maneuver and disruption with reinforcement learning approaches for multi-agent coordination

Asher, Derrik E.; Basak, Anjon; Fernández, R. Castillo; Sharma, Piyush; Zaroukian, Erin; Hsu, Christopher D.; Dorothy, Michael; Mahre, Thomas; Galindo, Géraldine; Frerichs, Luke; Rogers, John G.; Fossaceca, John M.

doi:10.1177/15485129221104096

Cited by 3 publications

(3 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been argued that a fundamental component of collaboration is the measurable interdependencies between agents performing a cooperative task, referred to as coordination. [1][2][3] Multi-agent tasks that require coordination typically result in the spatio-temporal alignment of actions, or simply coordinated behavior. 4 An example of multi-agent coordinated behavior relevant to this paper is the movement of robots across terrain in various geometric formations, which we refer to as formation control.…”

Section: Introductionmentioning

confidence: 99%

Coordinated maneuver in ground robotics for multiagent systems

Fernandez,

Adamson,

Luo

et al. 2024

Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications VI

View full text Add to dashboard Cite

Robots moving in formations can serve as the core functionality of coordinated maneuvers, which have realworld applications for logistics-based tasks, mobile asset protection, environmental monitoring, or wide-area surveillance. In order for autonomous multi-robot systems to be useful in such applications, they must be able to seamlessly execute actions, strategies, and behaviors in a coordinated manner. An example of coordinated behavior in multi-robot systems is the movement of robots across terrain in various geometric formations. The general goal of this work is to leverage formations inspired by military mobility and maneuver to guide the behaviors of autonomous multi-robot systems. Generally, our approach aims to develop coordinated multi-robot behaviors by utilizing domain knowledge to guide multi-agent systems. In this work, we measure a globalplanning based approach for control of a multi-robot system that is tasked with executing specific geometric formations in a Robot Operating System (ROS) based photo-realistic simulation environment (ARL Unity ROS Simulator (AURS)). This work highlights a quantitative assessment of global-planning based approaches for executing simple multi-robot coordinated behavior. Finally, the results of these experiments establish a baseline by providing metrics to evaluate a set of global navigation planners using a planning only approach to multi-agent coordinated maneuver.

show abstract

Section: Introductionmentioning

confidence: 99%

Coordinated maneuver in ground robotics for multiagent systems

Fernandez,

Adamson,

Luo

et al. 2024

Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications VI

View full text Add to dashboard Cite

show abstract

“…In recent years, Multi-Domain Operations (MDO) has become a critical operational concept since the multiple domains can be threatened by adversarial states or non-state actors, and the holistic control of the domains is essential for military vic-tory. Under MDO, a higher-level Command and Control (C2) is required to coordinate various resources across land, air, maritime, space, and cyberspace, and Artificial Intelligence (AI) has been recognized as the core component to support and guide the C2 in the process of OODA (Observe, Orient, Decide, Act) Loop 1,2 .…”

Section: Introductionmentioning

confidence: 99%

Military Decision Support with Actor and Critic Reinforcement Learning Agents

2024

Def. Sc. J.

View full text Add to dashboard Cite

While the recent advanced military operational concept requires an intelligent support of command and control, Reinforcement Learning (RL) has not been actively studied in the military domain. This study points out the limitations of RL for military applications from literature review and aims at improving the understanding of RL for military decision support under the limitations. Most of all, the black box characteristic of Deep RL makes the internal process difficult to understand in addition to complex simulation tools. A scalable weapon selection RL framework is built which can be solved either by a tabular form or a neural network form. The transition of the Deep Q-Network (DQN) solution to the tabular form makes it easier to compare the result to the Q-learning solution. Furthermore, rather than using one or two RL models selectively as before, RL models are divided as an actor and a critic, and systematically compared. A random agent, Q-learning and DQN agents as a critic, a Policy Gradient (PG) agent as an actor, Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) agents as an actor-critic approach are designed, trained, and tested. The performance results show that the trained DQN and PPO agents are the best decision supporter candidates for the weapon selection RL framework.

show abstract

“…In recent years, with the growing application of reinforcement learning (RL) theory in areas such as UAV search [11,12] , multi-agent coordination [13] , gaming [14] , robot autonomous navigation [15] , and other fields. Researchers have begun to combine DL and RL for object tracking.…”

mentioning

confidence: 99%

Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic Network

Xin¹,

Zhou²,

Hei³

et al. 2023

CAAI Artificial Intelligence Research

View full text Add to dashboard Cite

Aiming at the problem of poor tracking robustness caused by severe occlusion, deformation, and object rotation of deep learning object tracking algorithm in complex scenes, an improved deep reinforcement learning object tracking algorithm based on actordouble critic network is proposed. In offline training phase, the actor network moves the rectangular box representing the object location according to the input sequence image to obtain the action value, that is, the horizontal, vertical, and scale transformation of the object. Then, the designed double critic network is used to evaluate the action value, and the output double Q value is averaged to guide the actor network to optimize the tracking strategy. The design of double critic network effectively improves the stability and convergence, especially in challenging scenes such as object occlusion, and the tracking performance is significantly improved. In online tracking phase, the well-trained actor network is used to infer the changing action of the bounding box, directly causing the tracker to move the box to the object position in the current frame. Several comparative tracking experiments were conducted on the OTB100 visual tracker benchmark and the experimental results show that more intensive reward settings significantly increase the actor network's output probability of positive actions. This makes the tracking algorithm proposed in this paper outperforms the mainstream deep reinforcement learning tracking algorithms and deep learning tracking algorithms under the challenging attributes such as occlusion, deformation, and rotation.

show abstract

Strategic maneuver and disruption with reinforcement learning approaches for multi-agent coordination

Cited by 3 publications

References 47 publications

Coordinated maneuver in ground robotics for multiagent systems

Coordinated maneuver in ground robotics for multiagent systems

Military Decision Support with Actor and Critic Reinforcement Learning Agents

Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic Network

Contact Info

Product

Resources

About