Maneuver Decision-Making of Deep Learning for UCAV Thorough Azimuth Angles

Zhang, Hongpeng; Huang, Changqiang

doi:10.1109/access.2020.2966237

Cited by 20 publications

(12 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DRL has a distinct advantage over other approaches to flight control in its ability to perceive information and learn effectively. This makes it an ideal method for solving complex, high-dimensional sequential decisionmaking problems, such as those encountered in air combat [18]. In modern aerial combat with unmanned combat aerial vehicles (UCAVs), short-range aerial combat (dogfight) is still considered an important topic [19,20].…”

Section: Applications To Unmanned Combat Aerial Vehicles (Ucavs)mentioning

confidence: 99%

A deep reinforcement learning control approach for high-performance aircraft

De Marco,

D’Onza,

Manfredi

2023

Nonlinear Dyn

View full text Add to dashboard Cite

This research introduces a flight controller for a high-performance aircraft, able to follow randomly generated sequences of waypoints, at varying altitudes, in various types of scenarios. The study assumes a publicly available six-degree-of-freedom (6-DoF) rigid aeroplane flight dynamics model of a military fighter jet. Consolidated results in artificial intelligence and deep reinforcement learning (DRL) research are used to demonstrate the capability to make certain manoeuvres AI-based fully automatic for a high-fidelity nonlinear model of a fixed-wing aircraft. This work investigates the use of a deep deterministic policy gradient (DDPG) controller agent, based on the successful applications of the same approach to other domains. In the particular application to flight control presented here, the effort has been focused on the design of a suitable reward function used to train the agent to achieve some given navigation tasks. The trained controller is successful on highly coupled manoeuvres, including rapid sequences of turns, at both low and high flight Mach numbers, in simulations reproducing a prey–chaser dogfight scenario. Robustness to sensor noise, atmospheric disturbances, different initial flight conditions and varying reference signal shapes is also demonstrated.

show abstract

Section: Applications To Unmanned Combat Aerial Vehicles (Ucavs)mentioning

confidence: 99%

A deep reinforcement learning control approach for high-performance aircraft

De Marco,

D’Onza,

Manfredi

2023

Nonlinear Dyn

View full text Add to dashboard Cite

show abstract

“…To verify the real-time and accuracy of the algorithm, the robust maneuver decision theory [9] is selected from the maneuver decision methods based on the action library and choose the DNN decision method [36] and Bayesian decision method [16] in self-learning decision theory, compared with the method proposed in this study. In this section, the target aircraft is designed to make a large-radius serpentine maneuver at an altitude of 5000 meters, and UCAV uses four methods to pursue and analyze the situation of each method within the same decision step.…”

Section: Contrast Pursuitmentioning

confidence: 99%

Moving Time UCAV Maneuver Decision Based on the Dynamic Relational Weight Algorithm and Trajectory Prediction

Xie

Ding

Wei

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

To improve the accuracy and real-time performance of autonomous decision-making by the unmanned combat aerial vehicle (UCAV), a decision-making method combining the dynamic relational weight algorithm and moving time strategy is proposed, and trajectory prediction is added to maneuver decision-making. Considering the lack of continuity and diversity of air combat situation reflected by the constant weight in situation assessment, a dynamic relational weight algorithm is proposed to establish an air combat situation system and adjust the weight according to the current situation. Based on the dominance function, this method calculates the correlation degree of each subsituation and the total situation. According to the priority principle and information entropy theory, the hierarchical fitting function is proposed, the association expectation is calculated by using if-then rules, and the weight is dynamically adjusted. In trajectory prediction, the online sliding input module is introduced, and the long- and short-term memory (LSTM) network is used for real-time prediction. To further improve the prediction accuracy, the adaptive boosting (Ada) method is used to build the outer frame and compare with three traditional prediction networks. The results show that the prediction accuracy of Ada-LSTM is better. In the decision-making method, the moving time optimization strategy is adopted. To solve the problem of timeliness and optimization, each control variable is divided into 9 gradients, and there are 729 control schemes in the control sequence. Through contrast pursuit simulation experiments, it is verified that the maneuver decision method combining the dynamic relational weight algorithm and moving time strategy has a better accuracy and real-time performance. In the case of using prediction and not using prediction, the adaptive countermeasure simulation is carried out with the current more advanced Bayesian inference maneuvering decision-making scheme. The results show that the UCAV maneuvering decision-making ability combined with accurate prediction is better.

show abstract

“…Recently, some studies have used visibility graphs [29], Voronoi diagrams [30] and RL [31]- [34] in UAV path planning. Furthermore, some studies have applied RL to CUAV maneuvers [35]- [39]. However, the authors of those studies simply defined the state and action and conducted experiments in a dense reward environment.…”

Section: Related Workmentioning

confidence: 99%

Autonomous Control of Combat Unmanned Aerial Vehicles to Evade Surface-to-Air Missiles Using Deep Reinforcement Learning

Lee

Kim

2020

IEEE Access

View full text Add to dashboard Cite

This paper proposes a new reinforcement learning approach for executing combat unmanned aerial vehicle (CUAV) missions. We consider missions with the following goals: guided missile avoidance, shortestpath flight and formation flight. For reinforcement learning, the representation of the current agent state is important. We propose a novel method of using the coordinates and angle of a CUAV to effectively represent its state. Furthermore, we develop a reinforcement learning algorithm with enhanced exploration through amplification of the imitation effect (AIE). This algorithm consists of self-imitation learning and random network distillation algorithms. We assert that these two algorithms complement each other and that combining them amplifies the imitation effect for exploration. Empirical results show that the proposed AIE approach is highly effective at finding a CUAV's shortest-flight path while avoiding enemy missiles. Test results confirm that with our method, a single CUAV reaches its target from its starting point 95% of the time and a squadron of four simultaneously operating CUAVs reaches the target 70% of the time. INDEX TERMSDeep reinforcement learning, combat unmanned aerial vehicle, deep learning, autonomous flight management system, path planning, exploration

show abstract

Maneuver Decision-Making of Deep Learning for UCAV Thorough Azimuth Angles

Cited by 20 publications

References 20 publications

A deep reinforcement learning control approach for high-performance aircraft

A deep reinforcement learning control approach for high-performance aircraft

Moving Time UCAV Maneuver Decision Based on the Dynamic Relational Weight Algorithm and Trajectory Prediction

Autonomous Control of Combat Unmanned Aerial Vehicles to Evade Surface-to-Air Missiles Using Deep Reinforcement Learning

Contact Info

Product

Resources

About