Event-Triggered Communication Network With Limited-Bandwidth Constraint for Multi-Agent Reinforcement Learning

Abstract

Communicating with each other in a distributed manner and behaving as a group are essential in multi-agent reinforcement learning. However, real-world multi-agent systems suffer from restrictions on limited-bandwidth communication. If the bandwidth is fully occupied, some agents are not able to send messages promptly to others, causing decision delay and impairing cooperative effects. Recent related work has started to address the problem but still fails in maximally reducing the consumption of communication resources. In this paper, we propose Event-Triggered Communication Network (ETC-Net) to enhance the communication efficiency in multi-agent systems by sending messages only when necessary. According to the information theory, the limited bandwidth is translated to the penalty threshold of an event-triggered strategy, which determines whether an agent at each step sends a message or not. Then the design of the event-triggered strategy is formulated as a constrained Markov decision problem, and reinforcement learning finds the best communication protocol that satisfies the limited bandwidth constraint. Experiments on typical multi-agent tasks demonstrate that ETCNet outperforms other methods in terms of the reduction of bandwidth occupancy and still preserves the cooperative performance of multi-agent systems at the most.

Robust adaptive dynamic memory‐event‐triggered attitude control for nonlinear multi‐UAVs resist actuator hysteresis

“…Remark 2. Notice that conditions (11) and (12) of DMETM above contain historical information item 𝜂 i,j (t − T m ), which means that the trigger threshold is co-regulated by the current and historical dynamic information of auxiliary variables. In other words, the transmission frequency can be adjusted smoothly according to the dynamic characteristics of the system, which can reduce the impact on the system performance caused by the saltation of current data information to a certain extent.…”

“…The MASs are a class of very important large‐scale intelligent systems, which achieve the overall control goal through coordination among subsystems 10 . The communication bandwidth resource constraint is always an important factor affecting communication efficiency when the subsystems interact with each other through the communication network 11 . In the traditional time‐driven control framework, communication protocols generally transmit information at a predetermined time interval, which is undoubtedly conservative and will inevitably produce a large number of redundant data 12 .…”

“…In the traditional time‐driven control framework, communication protocols generally transmit information at a predetermined time interval, which is undoubtedly conservative and will inevitably produce a large number of redundant data 12 . Due to the limitation of network physical conditions, redundant data transmission will lead to communication channel overload and reduce communication efficiency 11 . Therefore, for MASs, it is very important to study how to reduce communication frequency and save limited communication resources.…”

“…10 The communication bandwidth resource constraint is always an important factor affecting communication efficiency when the subsystems interact with each other through the communication network. 11 In the traditional time-driven control framework, communication protocols generally transmit information at a predetermined time interval, which is undoubtedly conservative and will inevitably produce a large number of redundant data. 12 Due to the limitation of network physical conditions, redundant data transmission will lead to communication channel overload and reduce communication efficiency.…”

“…12 Due to the limitation of network physical conditions, redundant data transmission will lead to communication channel overload and reduce communication efficiency. 11 Therefore, for MASs, it is very important to study how to reduce communication frequency and save limited communication resources.…”