Multi-Agent DRL for Resource Allocation and Cache Design in Terrestrial-Satellite Networks

Li, Xiaonan; Zhang, Haijun; Zhou, Huan; Wang, Ning; Long, Keping; Al‐Rubaye, Saba; Karagiannidis, George K.

doi:10.1109/twc.2022.3231379

Cited by 14 publications

(2 citation statements)

References 41 publications

(40 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, there has been a surge of interest among various companies in providing widespread internet connectivity from space. This renewed focus gained momentum following the successful implementation of projects such as Starlink and OneWeb [7], [8].…”

Section: Background and Related Workmentioning

confidence: 99%

Future 6G Communications Powering Vertical Handover in Non-Terrestrial Networks

Warrier,

Aljaburi,

Whitworth

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

The integration of Unmanned Aerial Vehicles (UAVs) into future 6G networks will open new possibilities for applications ranging from surveillance to communication infrastructure maintenance, precision agriculture, and surveying. However, ensuring uninterrupted connectivity for UAVs operating in remote or dynamic environments remains a significant challenge. This paper presents a novel approach to achieving seamless handover for UAVs when transitioning between terrestrial and satellite communication networks. The proposed method in this paper, leverages graph theory and develop a decision-making algorithm to optimise handover decisions, minimizing latency, improving performance, and reducing service disruption. It establishes a comprehensive graph model that represents the dynamic topology of available network nodes, including terrestrial base stations and low earth orbit (LEO) satellites, which adapts in real-time to changes in UAV position and network conditions. The approach incorporates a decision-making algorithm that considers several factors, such as received signal strength (RSS), signal-to-noise ratio (SNR), and elevation angle, to determine the optimal time and location for a handover between terrestrial base stations and satellite links. This ensures a seamless transition between communication links, minimizing service disruption. The performance of this method is evaluated through extensive simulations and comparison with existing solutions demonstrating significant improvements in RSS, SNR, throughput, latency, ping-pongs and enhanced overall UAV connectivity. The proposed graph methodbased seamless handover solution represents a crucial advancement in enabling reliable and uninterrupted communication for UAVs operating in remote and challenging environments. By managing handovers between terrestrial and satellite networks, this research contributes to the realisation of the full potential of UAVs in emerging applications, thereby advancing the state-of-the-art in UAV technology.INDEX TERMS 6G, vertical handover, satellite, graph method, non-terrestrial networks (NTN) I. INTRODUCTION

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Future 6G Communications Powering Vertical Handover in Non-Terrestrial Networks

Warrier,

Aljaburi,

Whitworth

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…To enhance spectral efficiency and support massive connections, NOMA technology has attracted increasing attention in cache-assisted MEC networks. In [35], the multiagent deep deterministic policy gradient method was used to dynamically optimize the user association, power control and cache placement of BSs and satellites to improve the network energy efficiency in a NOMA-enabled satellite integrated with a terrestrial network scenario. In [36], joint optimization of offloading and caching decisions and computation resource allocation was performed to maximize long-term reward in cache-assisted NOMA-MEC networks under the predicted task popularity, and single-agent and multi-agent Q-learning algorithms were proposed to find feasible solutions.…”

Section: Related Workmentioning

confidence: 99%

Computing Offloading Based on TD3 Algorithm in Cache-Assisted Vehicular NOMA–MEC Networks

Zhou,

Xu,

Qin

et al. 2023

Sensors

View full text Add to dashboard Cite

In this paper, in order to reduce the energy consumption and time of data transmission, the non-orthogonal multiple access (NOMA) and mobile edge caching technologies are jointly considered in mobile edge computing (MEC) networks. As for the cache-assisted vehicular NOMA–MEC networks, a problem of minimizing the energy consumed by vehicles (mobile devices, MDs) is formulated under time and resource constraints, which jointly optimize the computing resource allocation, subchannel selection, device association, offloading and caching decisions. To solve the formulated problem, we develop an effective joint computation offloading and task-caching algorithm based on the twin-delayed deep deterministic policy gradient (TD3) algorithm. Such a TD3-based offloading (TD3O) algorithm includes a designed action transformation (AT) algorithm used for transforming continuous action space into a discrete one. In addition, to solve the formulated problem in a non-iterative manner, an effective heuristic algorithm (HA) is also designed. As for the designed algorithms, we provide some detailed analyses of computation complexity and convergence, and give some meaningful insights through simulation. Simulation results show that the TD3O algorithm could achieve lower local energy consumption than several benchmark algorithms, and HA could achieve lower consumption than the completely offloading algorithm and local execution algorithm.

show abstract

Traffic offloading and resource allocation enabled hybrid satellite‐terrestrial networks in cognitive integrated systems using a recurrent neuro‐fuzzy model

Mishra,

Mishra

2024

Satell Commun Network

View full text Add to dashboard Cite

SummaryIn recent years, the demand for high‐speed and reliable communication networks has grown exponentially. To meet this demand, researchers and engineers have been exploring innovative solutions that combine the benefits of both satellite and terrestrial networks. The complexity of accurately modeling and predicting dynamic network conditions to optimize resource distribution and maintain seamless connectivity. The objective of this work is to develop and implement a recurrent neuro‐fuzzy model (RNFM)for optimizing traffic offloading and resource allocation in hybrid satellite‐terrestrial networks within cognitive integrated systems. This work, begins with cognitive integrated hybrid satellite‐terrestrial networks employing spectrum‐sharing techniques. These techniques integrate with software‐defined networks (SDN) to facilitate traffic offloading in hybrid satellite‐terrestrial networks (H‐STN). The process incorporates a second‐price sealed‐bid auction mechanism to efficiently allocate resources. Joint resource allocation is then optimized using two convex optimization methods. This allocation, in turn, informs the resource allocation strategy. The system further incorporates user behavior analysis and employs a recurrent neuro‐fuzzy model with deep feed‐forward neural networks to enhance the accuracy and efficiency of the entire process. MATLAB simulation that incorporates adaptive learning algorithms and fuzzy logic to dynamically manage network resources and improve system efficiency. The findings show that the proposed technique outperforms both one‐step and multi‐step prediction algorithms with an accuracy increase of 99.23%. The future scope for this research is to integrate advanced machine learning algorithms, such as reinforcement learning, with the recurrent neuro‐fuzzy model to further enhance dynamic traffic offloading and resource allocation in increasingly complex and heterogeneous satellite‐terrestrial network environments.

show abstract

Multi-Agent DRL for Resource Allocation and Cache Design in Terrestrial-Satellite Networks

Cited by 14 publications

References 41 publications

Future 6G Communications Powering Vertical Handover in Non-Terrestrial Networks

Future 6G Communications Powering Vertical Handover in Non-Terrestrial Networks

Computing Offloading Based on TD3 Algorithm in Cache-Assisted Vehicular NOMA–MEC Networks

Traffic offloading and resource allocation enabled hybrid satellite‐terrestrial networks in cognitive integrated systems using a recurrent neuro‐fuzzy model

Contact Info

Product

Resources

About