Air Combat Strategy Using Approximate Dynamic Programming

Abstract

Unmanned Aircraft Systems (UAS) have the potential to perform many of the dangerous missions currently flown by manned aircraft. Yet, the complexity of some tasks, such as air combat, have precluded UAS from successfully carrying out these missions autonomously. This paper presents a formulation of a level flight, fixed velocity, one-on-one air combat maneuvering problem and an approximate dynamic programming (ADP) approach for computing an efficient approximation of the optimal policy. In the version of the problem formulation considered, the aircraft learning the optimal policy is given a slight performance advantage. This ADP approach provides a fast response to a rapidly changing tactical situation, long planning horizons, and good performance without explicit coding of air combat tactics. The method's success is due to extensive feature development, reward shaping and trajectory sampling. An accompanying fast and effective rollout based policy extraction method is used to accomplish on-line implementation. Simulation results are provided that demonstrate the robustness of the method against an opponent beginning from both offensive *

Guidance and control for own aircraft in the autonomous air combat: A historical review and future prospects

“…The major part was brought up in McGrew and How; 189 McGrew; 190 McGrew et al. 191 Later work that follows similar idea was found in Ma et al. 192 However, Ma et al.…”

“…

Figure 5.Model airplanes designed for the combat (above) and airplanes flown in the test (below). Source: reproduced with permission from Figure 14 in McGrew et al., 2010. 191

…”

“…192 However, Ma et al. 192 seems to be a direct implementation of the ADP algorithm illustrated in McGrew and How; 189 McGrew; 190 McGrew et al., 191 no theoretical breakthrough was found. Moreover, the authors analyzed simulation-based tests only to testify the algorithm, which (in our humble opinion) is not convincing enough, particularly as the original ADP development managed to realize real vehicle tests almost one decade prior to Ma et al.…”

“…To be precise, the ADP should be regarded as a problem solution scheme, not much a problem formulation. The Dynamic Programming (DP) can be used for solving the basic PE game, or as mentioned in McGrew and How; 189 McGrew; 190 and McGrew et al, 191 theoretically the ID-based problem formulation can be solved via DP. As a general-purpose planning algorithm, the DP accounts for various factors that may arise in the air combat problem-player's role, control sequences into the future, tactical advantage estimation, long planning horizon-practically everything that need/can be updated via progressive simulations.…”

“…The ADP approach developed in McGrew and How; 189 McGrew; 190 and McGrew et al starts with defining a baseline scoring function, which is similar to the utility as in ID, or the cost function in other approaches. It summarizes the tactical situation based on the relative heading and range between the two players.…”