A Pre-Trained Fuzzy Reinforcement Learning Method for the Pursuing Satellite in a One-to-One Game in Space

Wang, Xiao; Shi, Peng; Zhao, Yong; Sun, Yue

doi:10.3390/s20082253

Cited by 7 publications

(4 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The former problem is extensively solved by meta-heuristic algorithms [33,44]. The latter problem considers the requirement of tracking a non-cooperative target with maneuvering ability, and has been the subject of much recent attention [45]. The functions of both problems demonstrate strong nonlinearity and variable coupling, even without additional constraints.…”

Section: Applications Of the Proposed Methodsmentioning

confidence: 99%

Hierarchical Optimization Algorithm and Applications of Spacecraft Trajectory Optimization

Shi

Zhao

2022

Aerospace

Self Cite

View full text Add to dashboard Cite

The pursuit of excellent performance in meta-heuristic algorithms has led to a myriad of extensive and profound research and achievements. Notably, many space mission planning problems are solved with the help of meta-heuristic algorithms, and relevant studies continue to appear. This paper introduces a hierarchical optimization frame in which two types of particles—B-particles and S-particles—synergistically search for the optima. Global exploration relies on B-particles, whose motional direction and step length are designed independently. S-particles are for fine local exploitation near the current best B-particle. Two specific algorithms are designed according to this frame. New variants of classical benchmark functions are used to better test the proposed algorithms. Furthermore, two spacecraft trajectory optimization problems, spacecraft multi-impulse orbit transfer and the pursuit-evasion game of two spacecraft, are employed to examine the applicability of the proposed algorithms. The simulation results indicate that the hierarchical optimization algorithms perform well on given trials and have great potential for space mission planning.

show abstract

Section: Applications Of the Proposed Methodsmentioning

confidence: 99%

Hierarchical Optimization Algorithm and Applications of Spacecraft Trajectory Optimization

Shi

Zhao

2022

Aerospace

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, in the field of practical applications, RL is facing the challenges of low efficiency and hard convergence due to the large system state and decision-making state spaces [23]. In order to improve the practical application effect of reinforcement learning, there are factorial reinforcement learning (FRL) [24], hierarchical reinforcement learning (HRL) [25], inverse reinforcement learning (IRL) [26], deep reinforcement learning (DRL), etc. [27].…”

Section: Introductionmentioning

confidence: 99%

Accelerating Fuzzy Actor–Critic Learning via Suboptimal Knowledge for a Multi-Agent Tracking Problem

Wang

Mao

et al. 2023

Electronics

Self Cite

View full text Add to dashboard Cite

Multi-agent differential games usually include tracking policies and escaping policies. To obtain the proper policies in unknown environments, agents can learn through reinforcement learning. This typically requires a large amount of interaction with the environment, which is time-consuming and inefficient. However, if one can obtain an estimated model based on some prior knowledge, the control policy can be obtained based on suboptimal knowledge. Although there exists an error between the estimated model and the environment, the suboptimal guided policy will avoid unnecessary exploration; thus, the learning process can be significantly accelerated. Facing the problem of tracking policy optimization for multiple pursuers, this study proposed a new form of fuzzy actor–critic learning algorithm based on suboptimal knowledge (SK-FACL). In the SK-FACL, the information about the environment that can be obtained is abstracted as an estimated model, and the suboptimal guided policy is calculated based on the Apollonius circle. The guided policy is combined with the fuzzy actor–critic learning algorithm, improving the learning efficiency. Considering the ground game of two pursuers and one evader, the experimental results verified the advantages of the SK-FACL in reducing tracking error, adapting model error and adapting to sudden changes made by the evader compared with pure knowledge control and the pure fuzzy actor–critic learning algorithm.

show abstract

“…In recent years, the actor-critic algorithm has been attempted to solve some typical differential games under the unknown environment. [28][29][30][31] One of the typical games is the problem of territory guarding, which is a type of grid walking game on the ground. 32 In addition, the differential game between the pursuer and the evader with the single control input separately has been considered in References.…”

Section: Introductionmentioning

confidence: 99%

An algorithm of pretrained fuzzy actor–critic learning applying in fixed-time space differential game

Wang

Shi

Schwartz

et al. 2021

Proceedings of the Institution of Mechanical Engineers, Part G:

Self Cite

View full text Add to dashboard Cite

Solving space differential game in an unknown environment remains a challenging problem. This article proposes a pretrained fuzzy actor–critic learning algorithm for dealing with the space pursuit-evasion game in fixed time. It is supposed that the research objects are two agents including one pursuer and one evader in space. A virtual environment, which is defined as the known part of the real environment, is utilized for deriving optimal strategies of the pursuer and the evader, respectively. Through employing the fuzzy inference system, a pretrained process, which is based on the genetic algorithm, is designed to obtain the initial consequent set of the pursuer and the evader. Besides, an actor–critic framework is applied to finely learn the suitable consequent set of the pursuer and evader in the real environment. Numerical experimental results validate the effectiveness of the proposed algorithms on improving the ability of the agents to adapt to the real environment.

show abstract

A Pre-Trained Fuzzy Reinforcement Learning Method for the Pursuing Satellite in a One-to-One Game in Space

Cited by 7 publications

References 32 publications

Hierarchical Optimization Algorithm and Applications of Spacecraft Trajectory Optimization

Hierarchical Optimization Algorithm and Applications of Spacecraft Trajectory Optimization

Accelerating Fuzzy Actor–Critic Learning via Suboptimal Knowledge for a Multi-Agent Tracking Problem

An algorithm of pretrained fuzzy actor–critic learning applying in fixed-time space differential game

Contact Info

Product

Resources

About