Range-Aware Impact Angle Guidance Law With Deep Reinforcement Meta-Learning

Abstract

In this paper, a new guidance law is proposed for impact angle constrained missile with timevarying velocity against a maneuvering target. The proposed guidance law is based on model-based deep reinforcement learning (RL) technique where a deep neural network is trained to be a predictive model used in model predictive path integral (MPPI) control. Tube-MPPI, a robust approach utilizing ancillary controller for disturbance rejection, is introduced in guidance law design in this work to deal with the MPPI degradation of robustness when the deep predictive model differs with actual environment. To further improve the performance, meta-learning is utilized to enable the deep neural dynamics adapt to environment changes online. With this approach the model mismatch of the nominal controller is reduced to improve tube-MPPI performance. Furthermore, a range-aware hyperbolic function is proposed as an adaptive function in the MPPI performance index design. Thus, reduced initial acceleration command and increased terminal velocity benefit guidance performance. Numerical simulations under various conditions demonstrate the effectiveness of proposed guidance law. INDEX TERMS Missile guidance, tube model predictive control, meta-learning, deep reinforcement learning, impact angle constraint

Design of Three-Dimensional Intelligent Guidance Law for Intercepting Highly Maneuvering Target

“…In [19], a finite-time dual-layer guidance law considering the second-order autopilot dynamics was designed, and an extended state observers are used to estimate the unknown target maneuver and acceleration derivative of the missile. with the development of artificial intelligence technology, neural networks [20][21][22] and deep reinforcement learning technique [23][24][25][26][27][28] have also begun to be applied for the design of guidance laws to estimate the uncertain dynamics of maneuvering targets. Moreover, in order to reduce the computational cost of the neural networks, a non-fragile quantitative prescribed performance control method was developed in [29], a simplified finite-time fuzzy neural controller has been introduced in [30], an adaptive critic design-based fuzzy neural controller was proposed in [31].…”

Hierarchical reinforcement learning guidance with threat avoidance

“…Different from the UAVs, the speed of missile is faster and the direction is more difficult to control [19,20]. An adaptive guidance system using the reinforcement meta-learning with recurrent network was proposed in [21]. A model-based DRL method was presented in [22] to predict the model of the guidance dynamics, and the predicted result was incorporated into a model predictive path integral control framework.…”

Cooperative guidance of multiple missiles: a hybrid co-evolutionary approach

“…Some of these algorithms have achieved satisfactory results in coping with problems with low complexity and accuracy requirements. In [4], [5] and [13], the state-of-the-art reinforcement learning frameworks have demonstrated their effectiveness in the guidance task. Zhang et.al proposed a gradient-descent-based reinforcement learning method in the actor-critic framework and achieved consensus control for multiagent systems by following a tracking leader [14].…”