Deep Reinforcement Learning Based QoS-Aware Routing in Knowledge-Defined Networking

Abstract

Knowledge-Defined networking (KDN) is a concept that relies on Software-Defined networking (SDN) and Machine Learning (ML) in order to operate and optimize data networks. Thanks to SDN, a centralized path calculation can be deployed, thus enhancing the network utilization as well as Quality of Services (QoS). QoS-aware routing problem is a high complexity problem, especially when there are multiple flows coexisting in the same network. Deep Reinforcement Learning (DRL) is an emerging technique that is able to cope with such complex problem. Recent studies confirm the ability of DRL in solving complex routing problems; however, its performance in the network with QoSsensitive flows has not been addressed. In this paper, we exploit a DRL agent with convolutional neural networks in the context of KDN in order to enhance the performance of QoS-aware routing. The obtained results demonstrate that the proposed approach is able to improve the performance of routing configurations significantly even in complex networks.

A Deep Reinforcement Learning Approach for VNF Forwarding Graph Embedding

“…The majority have been interested in the issue of routing [29], which is a sub-problem of VNF-FG placement.…”

AI-Assisted Framework for Green-Routing and Load Balancing in Hybrid Software-Defined Networking: Proposal, Challenges and Future Perspective

“…Pham et al [130] proposed an SDN-enabled QoS-aware routing framework that employs DRL with CNN. This study was undertaken in the scope of knowledge defines networking (KDN).…”

“…Based on the supervised ML module, LSTM-RNN algorithm is employed to extract shortterm network data traffic variabilities and periodicities, resulting in the meaningful features which are combined at the integration step to ensure traffic flow prediction and energy-efficient routing with guaranteed QoS performance. On the other hand, the DRL module performs learning from the existing historical data or right from scratch by iteratively interfacing with the defined network setting [114][115][116][117][118] [130]. Using publicly available dataset, the module can be evaluated in terms of accuracy and convergence speed.…”

Multi-domain non-cooperative VNF-FG embedding: A deep reinforcement learning approach

“…where φ l l is a binary variable which is 1 if l is deployed at the substrate link l, r l,bw is the available amount of bandwidth at the substrate link l, and D(φ l ) and R(φ l ) are the actual latency and loss rate corresponding to a given mapping [20]), this form of VNF-FG requests enables us to apply convolutional layers to learn the mutual impacts between VLs.…”

“…In [20], the continuous link weights were adopted as the action space of routing problems since it reduces the size of action space to |L| and improves the performance of learning process. Then, the deep deterministic policy gradient (DDPG) method was utilized to deal with continuous action space thanks to its performance [21].…”