Fast-Convergence Reinforcement Learning for Routing in LEO Satellite Networks

Abstract

Fast convergence routing is an important issue for LEO constellation network, due to its dynamical topology changing and time varying transmission requests. Most of existing research focus on the OSPF routing algorithm, which cannot handle the frequently links state changing of network. In this paper, we propose a Fast-Convergence Reinforcement Learning Satellite Routing Algorithm (FRL-SR) for LEO satellite networks, in which the satellite gets the network links status fast and adjusts its routing strategy. In FRL-SR, each satellite node is regarded as an agent. The agent selects the port for packet forwarding according to its own routing policy. When the satellite network state changes, agent would send ’hello’ packets to the neighbor node to update the neighbor node’s routing policy. Compared with traditional reinforcement learning, FRL-SR can perceive network information faster, and then converge faster. Also, FRL-SR can mask the dynamics of satellite network topology and adaptively adjust the forwarding strategy according to the link state. Various simulation is constructed, the results show that the proposed FRL-SR algorithm out performance the Dijkstra algorithm in performance of average delay, packet arriving ratio, network load balance.