A Routing Optimization Method for Software-Defined SGIN Based on Deep Reinforcement Learning

Tu, Zhe; Zhou, Huachun; Li, Kun; Li, Guanglei; Shen, Qi

doi:10.1109/gcwkshps45667.2019.9024680

Cited by 17 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The more traditional method Kmeans is also used with medium-size databases [111], [112]. Similar to supervised learning, deep reinforcement learning should be applied in those scenarios where multiple iterations with the environment are permitted, specially the LSTMs and RNNs [113]- [115]. Neural networks need to be extensively trained.…”

Section: Selecting the Best ML Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Machine Learning Techniques for Routing Optimization in SDN

et al. 2021

View full text Add to dashboard Cite

In conventional networks, there was a tight bond between the control plane and the data plane. The introduction of Software-Defined Networking (SDN) separated these planes, and provided additional features and tools to solve some of the problems of traditional network (i.e., latency, consistency, efficiency). SDN is a flexible networking paradigm that boosts network control, programmability and automation. It proffers many benefits in many areas, including routing. More specifically, for efficiently organizing, managing and optimizing routing in networks, some intelligence is required, and SDN offers the possibility to easily integrate it. To this purpose, many researchers implemented different machine learning (ML) techniques to enhance SDN routing applications. This article surveys the use of ML techniques for routing optimization in SDN based on three core categories (i.e. supervised learning, unsupervised learning, and reinforcement learning). The main contributions of this survey are threefold. Firstly, it presents detailed summary tables related to these studies and their comparison is also discussed, including a summary of the best works according to our analysis. Secondly, it summarizes the main findings, best works and missing aspects, and it includes a quick guideline to choose the best ML technique in this field (based on available resources and objectives). Finally, it provides specific future research directions divided into six sections to conclude the survey. Our conclusion is that there is a huge trend to use intelligence-based routing in programmable networks, particularly during the last three years, but a lot of effort is still required to achieve comprehensive comparisons and synergies of approaches, meaningful evaluations based on open datasets and topologies, and detailed practical implementations (following recent standards) that could be adopted by industry. In summary, future efforts should be focused on reproducible research rather than on new isolated ideas. Otherwise, most of these applications will be barely implemented in practice.

show abstract

Section: Selecting the Best ML Methodsmentioning

confidence: 99%

“…Tu et al [115] highlight the existing challenge for optimized routing in space-ground integration networks, particularly when changes occur in the topology and link status. For that purpose, they define the ML-SSGIN framework, which uses the DDPG algorithm and a a neural network that integrates LSTM and Dense layers.…”

Section: ) Deep Reinforcement Learningmentioning

confidence: 99%

A Survey on Machine Learning Techniques for Routing Optimization in SDN

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The algorithm finds the shortest communication path between satellites and optimizes the routing in real-time. The work in [64] combined machine learning and an SDN to solve dynamic network topology and link traffic awareness in multi-layer satellite networks. The scheme adopts the deep deterministic policy gradient (DDPG) algorithm for routing optimization.…”

Section: Sdn-based Dynamic Routings In Satellite Networkmentioning

confidence: 99%

Dynamic Routings in Satellite Networks: An Overview

Cao

Xiong

et al. 2022

Sensors

View full text Add to dashboard Cite

The Satellite network is an important part of the global network. However, the complex architecture, changeable constellation topology, and frequent inter-satellite connection switching problems bring great challenges to the routing designs of satellite networks, making the study of the routing methods in satellite networks a research hotspot. Therefore, this paper investigates the latest existing routing works to tackle the dynamic routing problems in satellite networks. The architecture and development of satellite networks are presented and analyzed first. Afterward, dynamic routing problems in satellite networks are analyzed in detail based on the time-varying network topology. According to the latest works, the advanced satellite network routing schemes, including single-layer and multi-layer dynamic routing are introduced and analyzed. In addition, the merits, shortcomings, and applications of these schemes are analyzed and summarized. Finally, potential technologies and future directions are discussed.

show abstract

“…

where

, and

are balance weights [ 40 ]. A software-defined space–ground-integration-network-based deep reinforcement learning algorithm was proposed, which computes reward r to evaluate the performance of a link as

where

, and

are adjustment factors [ 41 ]. A priority-and-failure-probability-based (PFPR) routing was proposed to fulfill the QoS requirements of different services in LEO/MEO satellite communication links using an objective function

that uses a linear sum as below.…”

Section: Related Workmentioning

confidence: 99%

Delay-Packet-Loss-Optimized Distributed Routing Using Spiking Neural Network in Delay-Tolerant Networking

Velusamy

Lent

2022

Sensors

View full text Add to dashboard Cite

Satellite communication is inevitable due to the Internet of Everything and the exponential increase in the usage of smart devices. Satellites have been used in many applications to make human life safe, secure, sophisticated, and more productive. The applications that benefit from satellite communication are Earth observation (EO), military missions, disaster management, and 5G/6G integration, to name a few. These applications rely on the timely and accurate delivery of space data to ground stations. However, the channels between satellites and ground stations suffer attenuation caused by uncertain weather conditions and long delays due to line-of-sight constraints, congestion, and physical distance. Though inter-satellite links (ISLs) and inter-orbital links (IOLs) create multiple paths between satellite nodes, both ISLs and IOLs have the same issues. Some essential applications, such as EO, depend on time-sensitive and error-free data delivery, which needs better throughput connections. It is challenging to route space data to ground stations with better QoS by leveraging the ISLs and IOLs. Routing approaches that use the shortest path to optimize latency may cause packet losses and reduced throughput based on the channel conditions, while routing methods that try to avoid packet losses may end up delivering data with long delays. Existing routing algorithms that use multi-optimization goals tend to use priority-based optimization to optimize either of the metrics. However, critical satellite missions that depend on high-throughput and low-latency data delivery need routing approaches that optimize both metrics concurrently. We used a modified version of Kleinrock’s power metric to reduce delay and packet losses and verified it with experimental evaluations. We used a cognitive space routing approach, which uses a reinforcement-learning-based spiking neural network to implement routing strategies in NASA’s High Rate Delay Tolerant Networking (HDTN) project.

show abstract

A Routing Optimization Method for Software-Defined SGIN Based on Deep Reinforcement Learning

Cited by 17 publications

References 11 publications

A Survey on Machine Learning Techniques for Routing Optimization in SDN

A Survey on Machine Learning Techniques for Routing Optimization in SDN

Dynamic Routings in Satellite Networks: An Overview

Delay-Packet-Loss-Optimized Distributed Routing Using Spiking Neural Network in Delay-Tolerant Networking

Contact Info

Product

Resources

About