State-constrained agile missile control with adaptive-critic-based neural networks

Abstract

In this study, we develop an adaptive-critic-based controller to steer an agile missile that has a constraint on the minimum flight Mach number from various initial Mach numbers to a given final Mach number in minimum time while completely reversing its flightpath angle. This class of bounded state space, free final time problems is very difficult to solve due to discontinuities in costates at the constraint boundaries.We use a two-neural-network structure called "adaptive critic" in this study to carry out the optimization process. This structure obtains an optimal controller through solving optimal control-related equations resulting from a Hamiltonian formulation. Detailed derivations of equations and conditions on the constraint boundary are provided. For numerical experiments, we consider vertical plane scenarios. Flight Mach number and the flightpath angle are the states and the aerodynamic angle of attack is treated as the control. Numerical results bring out some attractive features of the adaptive critic approach and show that this formulation works very well in guiding the missile to its final conditions for this state constrained optimization problem from an envelope of initial conditions.

Online reinforcement learning for fixed-wing aircraft longitudinal control

“…One of the first implementation cases for linear aircraft control simulation can be found in [18]. For nonlinear flight control, successful implementation cases have been reported for a full scale model of: missile [19,20], fixed-wing aircraft [21][22][23][24][25][26], and helicopter [27].…”

Adaptive Critic-Based Solution to an Orbital Rendezvous Problem

“…where V k ∈ R r×m and W k ∈ R s are the weights of the actor and critic networks, respectively, at time step k. The basis functions are given by θ∶ R n × R l → R r and η∶ R n × R l → R s for r and s being positive integers, denoting the number of neurons in the respective network. The selected form for approximate optimal cost-to-go (12) is motivated by the assumed representation for the cost-to-go in the linear problem [i.e., Eq. (8)], adapted from [10].…”

“…For this purpose, the gradient of J k , given by Eq. (12), with respect to v, is calculated and used in Eq. (9) to obtain…”

“…In case of divergence, using higher order terms in the expansion for accuracy of the result may not help. The use of intelligent control for solving finite horizon optimal control problems was considered in [12][13][14], among which [13] solves the problems with soft terminal constraint. The controllers developed in [12,14] admit hard terminal constraints.…”

“…However, the ideas in these papers cannot be used in the rendezvous problem. In [12], an intelligent controller is developed for an agile missile maneuver. It is, however, a scalar problem, wherein the final state and the control have a direct relationship.…”