An Adaptive Fuzzy Reinforcement Learning Cooperative Approach for the Autonomous Control of Flock Systems

Abstract

The flock-guidance problem enjoys a challenging structure where multiple optimization objectives are solved simultaneously. This usually necessitates different control approaches to tackle various objectives, such as guidance, collision avoidance, and cohesion. The guidance schemes, in particular, have long suffered from complex tracking-error dynamics. Furthermore, techniques that are based on linear feedback strategies obtained at equilibrium conditions either may not hold or degrade when applied to uncertain dynamic environments. Pre-tuned fuzzy inference architectures lack robustness under such unmodeled conditions. This work introduces an adaptive distributed technique for the autonomous control of flock systems. Its relatively flexible structure is based on online fuzzy reinforcement learning schemes which simultaneously target a number of objectives; namely, following a leader, avoiding collision, and reaching a flock velocity consensus. In addition to its resilience in the face of dynamic disturbances, the algorithm does not require more than the agent position as a feedback signal. The effectiveness of the proposed method is validated with two simulation scenarios and benchmarked against a similar technique from the literature.

Real-time measurement-driven reinforcement learning control approach for uncertain nonlinear systems

“…These approaches have been used to solve cooperative control problems for multi-agent systems communicating over graphs [39][40][41]. An adaptive Fuzzy-RL mechanism is adopted to control flocking motion of a swarm of robots in [42]. Regression models such as iterative and batch least squares are employed to implement the PI solutions [37,43].…”

Communication-free shepherding navigation with multiple steering agents

“…Shepherding problem (Long et al, 2020) refers to the problem of designing the movement law of steering agents (called shepherds) to navigate another set of agents (called sheep) driven by the repulsive force from steering agents and has been attracting emerging attention by its applicability in robotics (Chung et al, 2018), group dynamics (Vemula et al, 2018), and nanochemistry (Mou et al, 2020). Against the development of leader-following control (Consolini et al, 2008;Qu et al, 2021), several works of shepherding research have been published toward providing effective solutions to the shepherding problem within various scientific fields including the systems and control theory (Bacon and Olgac, 2012;Pierson and Schwager, 2018), robotics (Zhi and Lien, 2021), and the complexity science (El-Fiqi et al, 2020).…”

T3OMVP: A Transformer-based Time and Team Reinforcement Learning Scheme for Observation-constrained Multi-Vehicle Pursuit in Urban Area

“…However, it becomes challenging for game theory solutions to define a suitable objective function when the problem becomes more complex, such as the increased number of agents, restricted movement, and complicated environment. With respect to cooperative MARL research for the MVP game, the multi-agent system is modeled using Markov decision processes (MDP) [9], and a neural network can be used to approximate the complex objective function [10]. Cristino et al used the Twin Delayed Deep Deterministic Policy Gradient (TD3) to demonstrate a real-world pursuit-evasion in open environment with boundaries [11].…”