Self-learning Routing for Optical Networks

Abstract

It is generally very difficult to optimize the routing policies in optical networks with dynamic traffic. Most widely-used routing policies, e.g., shortest path routing and least congested path (LCP) routing, are heuristic policies. Although the LCP is often regarded as the bestperforming adaptive routing policy, we are often eager to know whether there exist better routing policies that surpass these heuristics in performance. In this paper, we propose a framework of reinforcement learning (RL) based routing scheme, that learns routing decisions during the interactions with the environment. With a proposed self-learning method, the RL agent can improve its routing policy continuously. Simulations on a ring-topology metro optical network demonstrate that, the proposed scheme outperforms the LCP routing policy.

Deep Reinforcement Learning-Based RMSA Policy Distillation for Elastic Optical Networks

“…There are some other works exploring various aspects by applying DRL in the optical network management. Huang et al [15] proposed a DRL-based self-learning routing scheme for the WDM-based networks. It allows the agent to continuously improve its performance by self-comparison.…”

“…The value network outputs a value V(s t ; θ v,T ), which is a real number. Finally, we store the sample (s t , a t , r t , V(s t ; θ v,T )) generated by the interaction of the agent and the environment in an experience buffer D. When the size of experience buffer reaches 2N − 1, we perform training based on the first N samples (lines [13][14][15][16][17][18][19]. For each sample at time t, the advantage function is calculated in line 15.…”

“…However, these rule-based heuristics, mostly relying on researchers' cognition, cannot comprehensively capture the effect of the complex network conditions. To overcome the above limitation, deep reinforcement learning (DRL) has recently been introduced to the RMSA problem [14][15][16][17][18][19], where the RMSA policies are parameterized by deep neural networks and the RMSA policies are improved through interactions with the optical network environment. Many of them have achieved a better performance than heuristic methods.…”