Multi-agent deep learning for simultaneous optimization for time and energy in distributed routing system

Abstract

Packet routing problem most commonly emerges in the context of computer networks, thus the majority of routing algorithms existing nowadays is designed specifically for routing in computer networks. However, in the logistics domain, many problems can be formulated in terms of packet routing, e.g. in automated traffic routing or material handling systems. In this paper, we propose an algorithm for packet routing in such heterogeneous environments. Our approach is based on deep reinforcement learning networks combined with link-state protocol and preliminary supervised learning. Similarly to most routing algorithms, the proposed algorithm is a distributed one and is designed to run on a network constructed from interconnected routers. Unlike most other algorithms, proposed one views routers as learning agents, representing the routing problem as a multi-agent reinforcement learning problem. Modeling each router as a deep neural network allows each router to account for heterogeneous data about its environment, allowing for optimization of more complex cost functions, like in case of simultaneous optimization of bag delivery time and energy consumption in a baggage handling system. We tested the algorithm using manually constructed simulation models of computer network and baggage handling system. It outperforms state-of-the-art routing algorithms.

QLGR: A Q-learning-based Geographic FANET Routing Algorithm Based on Multi-agent Reinforcement Learning

“…The literature [31] applies the DQN-routing algorithm in Deep Reinforcement Learning DRL to solve the routing problem, which combines the advantages of Q-routing and DQN. Each router is considered as an agent whose parameters are shared and updated simultaneously during the training process (centralized training), but it provides independent packet transmission instructions (decentralized execution).…”

“…Energy is an important factor in UAV scenarios. To examine the energy consumption of the routing protocol, we counted the residual energy as a performance parameter, which is calculated as shown in (31). R is the number of rounds of UAV executing tasks, we set the amount of data to be distributed to execute one round of tasks to 1000 / bit round , the communication distance threshold 0 300…”

MAMRL: Exploiting Multi-agent Meta Reinforcement Learning in WAN Traffic Engineering

“…This prevents the straightforward use of experience replay, which is crucial for stabilizing deep Q learning [34]. [35] combines the Q-routing and deep Q-learning to solve the routing problem. However, the training process of the algorithm proposed in [35] is in a centralized manner (all the routers need to share parameters), which might cause issues in real-world large-scale network environments.…”

“…[35] combines the Q-routing and deep Q-learning to solve the routing problem. However, the training process of the algorithm proposed in [35] is in a centralized manner (all the routers need to share parameters), which might cause issues in real-world large-scale network environments. The authors in [36] propose to use a deep actor-critic reinforcement learning algorithm to optimize the performance of the communication network.…”

A Pointer Neural Network for the Vehicle Routing Problem with Task Priority and Limited Resources

“…In recent years, with the rapid development and successful application of machine learning technology in various fields, such as management operation research [10,16], medicine [4,22], computer science [11], etc., several technologies have been introduced to solve combinatorial optimization problems [26,29]. Vinyals et al [29] proposed a model consisting of two recurrent neural networks (RNNs) and an attention mechanism to solve combinatorial optimization problems.…”