Maneuvering penetration strategies of ballistic missiles based on deep reinforcement learning

Abstract

In this paper, a ballistic missile terminal penetration scenario is studied, which contains three participants: target, missile, and defender. The ballistic missile attempts to hit the target while evading the defender. A maneuvering penetration guidance strategy that balances both the guidance accuracy and penetration capability is proposed through deep reinforcement learning. Reward shaping and random initialization are applied to improve training speed and generalization, respectively. The proposed strategy is developed based on the twin delayed deep deterministic policy gradient algorithm. It directly maps observations to actions and is an end-to-end guidance scheme that does not require an accurate model. The simulation results show that the proposed strategy has higher penetration probabilities than conventional strategies for different initial heading errors and even for defenders with different guidance laws, which indicates its good robustness and generalization. For different initial heading errors, it has learned different maneuvering modes and has certain intelligence. In addition, it is computationally small, does not consume much memory, and can be easily applied on modern flight computers.

Calculate the ignition height of the vertical landing phase online for the reusable rocket

“…21,22 Maneuvring penetration strategies of ballistic missiles are proposed based on DRL. 23 A guidance strategy for spacecraft proximity operations is proposed based on DRL. 24 So, DRL has a powerful capability of solving problems with uncertainties offline.…”

Intelligent maneuver strategy for hypersonic vehicles in three-player pursuit-evasion games via deep reinforcement learning

“…The trust region policy optimization (TRPO) algorithm was proposed to generate an interception guidance law ( Chen et al, 2023 ). With an emphasis on the terminal evasion scenario, Qiu et al (2022) , based on DRL, developed a maneuver evasion guidance method that took into account both guidance accuracy and evasion capabilities. In a different study ( Jiang et al, 2022 ), the problem was reformulated as a Markov decision process (MDP), and an Actor-Critic (AC) framework-based DRL algorithm was used to solve it to suggest the anti-interception guiding law.…”

Intelligent Maneuver Strategy for a Hypersonic Pursuit-Evasion Game Based on Deep Reinforcement Learning

“…The intelligent game maneuver adopts a closed-loop maneuver scheme of "interceptor movement-situational awareness-maneuver strategy generation-maneuver control implementation" that realizes timely maneuvering to increase miss distance and increase evasion probability. The key to intelligent game maneuver lies in the selection of intelligent algorithms Among the intelligent algorithms associated with hypersonic aircraft, deep learning (DL)and reinforcement learning (RL) are the first to bear the brunt [21][22][23][24][25][26][27][28][29][30][31][32]. Due to its strong nonlinear fitting ability, the deep neural network (DNN) in DL has been widely used in the PE problems of hypersonic aircraft [21][22][23].…”

“…Among these, the most prevalent study [21] resolves the tension between the accuracy and speed of the IPP by building an IPP neural network model after using the ballistic model to create training data. And the algorithms of reinforcement learning, especially deep reinforcement learning (DRL), provide a new approach to the design of HVs' evasion strategies [24][25][26][27][28][29][30][31][32]. As an unsupervised heuristic algorithm without an accurate model, RL and DRL can generate actions based on the interaction with the environment, that is, conduct intelligent maneuvering games based on both attack and defense sides.…”