BeiDou Short-Message Satellite Resource Allocation Algorithm Based on Deep Reinforcement Learning

Abstract

The comprehensively completed BDS-3 short-message communication system, known as the short-message satellite communication system (SMSCS), will be widely used in traditional blind communication areas in the future. However, short-message processing resources for short-message satellites are relatively scarce. To improve the resource utilization of satellite systems and ensure the service quality of the short-message terminal is adequate, it is necessary to allocate and schedule short-message satellite processing resources in a multi-satellite coverage area. In order to solve the above problems, a short-message satellite resource allocation algorithm based on deep reinforcement learning (DRL-SRA) is proposed. First of all, using the characteristics of the SMSCS, a multi-objective joint optimization satellite resource allocation model is established to reduce short-message terminal path transmission loss, and achieve satellite load balancing and an adequate quality of service. Then, the number of input data dimensions is reduced using the region division strategy and a feature extraction network. The continuous spatial state is parameterized with a deep reinforcement learning algorithm based on the deep deterministic policy gradient (DDPG) framework. The simulation results show that the proposed algorithm can reduce the transmission loss of the short-message terminal path, improve the quality of service, and increase the resource utilization efficiency of the short-message satellite system while ensuring an appropriate satellite load balance.

Deep Deterministic Policy Gradient Algorithm: A Systematic Review

“…DDPG is widely employed for resource allocation problems in cloud computing [33] and energy management [34] to optimize resource allocation, such as computing resources, memory, storage, and bandwidth. In cloud computing, DDPG can be used to allocate virtual machines to different physical servers based on the current load and demand to ensure efficient utilization of the resources and minimize the cost.…”

“…Hybrid Electric Tracked Vehicle (SHETV) (Xia et al, 2021) [33] Techniques: DRL algorithm based on (DDPG) framework. Methodology: A model for satellite resource allocation is developed that optimizes multiple objectives.…”

A Word-Granular Adversarial Attacks Framework for Causal Event Extraction

“…It currently performs well on many decision-based problems. For example, in games or other fields, it has many applications [ 21 , 22 , 23 ]. At the same time, we believe that the application of reinforcement learning to NLP still has great potential.…”

An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning

“…It can effectively solve difficult sample data acquisition problems in the constellation’s early warning system by continuously updating its decision network via the interaction between the intelligence and the environment. Commonly used deep-reinforcement-learning algorithms currently include the following: deep Q-network (DQN) [ 17 , 18 , 19 , 20 ], deep deterministic policy gradient (DDPG) [ 21 , 22 , 23 ], proximal policy optimization (PPO) [ 24 , 25 , 26 ], and soft actor–critic (SAC). These algorithms are widely used in the cooperative positioning of moving targets, agile Earth observation satellite mission scheduling, and the collaborative scheduling of ground satellites.…”