Reinforcement Learning-Based Optimal Flat Spin Recovery for Unmanned Aerial Vehicle

Kim, Donghae; Oh, Gyeongtaek; Seo, Yongjun; Kim, Youdan

doi:10.2514/1.g001739

Cited by 30 publications

(13 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on our review of the related research and cited papers, it has been assessed that in spite of being an active area of research, application of RL for UAV application is still in its infancy. The applications are focused towards limited segments mainly handling segmented flight phases [34,35]. Keeping in view the immense potential of RL algorithms and its limited application in entirety for UAV Flight Control systems development, it is considered mandatory to explore this dimension.…”

Section: Relevant Studiesmentioning

confidence: 99%

Deep Reinforcement Learning for Integrated Non-Linear Control of Autonomous UAVs

et al. 2022

View full text Add to dashboard Cite

In this research, an intelligent control architecture for an experimental Unmanned Aerial Vehicle (UAV) bearing unconventional inverted V-tail design, is presented. To handle UAV’s inherent control complexities, while keeping them computationally acceptable, a variant of distinct Deep Reinforcement Learning (DRL) algorithm, namely Deep Deterministic Policy Gradient (DDPG) is proposed. Conventional DDPG algorithm after being modified in its learning architecture becomes capable of intelligently handling the continuous state and control space domains besides controlling the platform in its entire flight regime. Nonlinear simulations were then performed to analyze UAV performance under different environmental and launch conditions. The effectiveness of the proposed strategy is further demonstrated by comparing the results with the linear controller for the same UAV whose feedback loop gains are optimized by employing technique of optimal control theory. Results indicate the significance of the proposed control architecture and its inherent capability to adapt dynamically to the changing environment, thereby making it of significant utility to airborne UAV applications.

show abstract

Section: Relevant Studiesmentioning

confidence: 99%

Deep Reinforcement Learning for Integrated Non-Linear Control of Autonomous UAVs

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Behavior modelling and activity interpretation are of increasing interest in the information society [5]. The research on behavior cognition mainly focuses on computer science and network and social psychology, and the research targets mainly include human [6], animal [7], traffic [8], [9] and robot [10]. The Google team proposed in 2006 that the motion behavior cognition system should be composed of four modules of ''sensor-identification-transformation-controlled system (SITR)'' [11].…”

Section: A Behavior Cognitionmentioning

confidence: 99%

Communication Emitter Motion Behavior’s Cognition Based on Deep Reinforcement Learning

Wang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Considering the successful application of deep reinforcement learning (DRL) on tasks of moving objects, this paper innovatively applies deep deterministic policy gradient algorithm (DDPG) to complete the cognition task on multi-dimension and continuous communication emitter motion behavior. First, we propose a DDPG-based behavior cognition algorithm (DDPG-BC). It chooses direction, velocity, and communication frequency as state space, gains experience from interaction between network and environment and outputs deterministic cognition results. Second, under the condition of sufficient prior information such as geographic information, we further propose a novel algorithm named DDPG-based behavior cognition with Attention algorithm (DDPG+A-BC). It introduces attention mechanism into DDPG-BC which limits exploration scope and the randomness of initial state and improves the exploration efficiency and accuracy. The simulation experiments verify that DDPG-BC and DDPG+A-BC show good cognition ability on two different data set. And the algorithms are all superior to other DRL algorithm and existing cognition method with higher cognition accuracy and less time. In addition, we also discuss the influence of episode, reward function, and added attention mechanism on algorithm performance.

show abstract

“…Both pitch vectoring and yaw vectoring were demonstrated to aid recovery from flat spins (Planeaux, 1991) High performance fighter Following two controllers were proposed to effectively recover a generic fighter configurations from flat spins: Positional recovery controller that automatically sense flat spin and apply pro-spin aileron and anti-spin rudder. Once the rotation is stopped elevator is applied to bring aircraft in low alpha flight regime Pitch rocking controller that deflects elevators to maximum limits in a direction that depends upon the sign of the sensed pitch rate (elevator fully down when positive pitch rate is sensed and vice versa) (Goman et al, 2005) UAV Reinforcement learning (RL)-based optimal controller was successfully demonstrated to recover a UAV from a stable flat spin mode to low alpha flight regime in minimum time (Kim et al, 2016) HARV Problem of aircraft spin recovery is solved as a trajectory optimization problem using direct multiple shooting method with time and altitude-loss as cost functions to be minimized (Rao and Go, 2019) into fully developed spin. Formulating spin recovery procedure for the contemporary fighter configurations, which mostly experience fast oscillatory spins, is a challenging task, as it requires counterintuitive control inputs.…”

Section: Automatic Spin Recovery Control Systemsmentioning

confidence: 99%

A review and historical development of analytical techniques to predict aircraft spin and recovery characteristics

Malik

Masud

Akhtar

2020

AEAT

View full text Add to dashboard Cite

Purpose This paper aims to present a literature review on analytical research on the prediction of aircraft spin and recovery characteristics, as it progressed from the early years of aviation to current state of the art spin technologies. Design/methodology/approach Aerodynamic model development approaches that have been generally used in past spin studies are presented. Past contributions in application of these analytical techniques to predict spin and recovery characteristics on various fighters, general aviation and airliners are discussed, thus providing useful reference for researchers embarking aircraft spin research. An overview of the development of spin prevention and spin recovery technologies to mitigate stall/spin susceptibility is presented. Findings The challenges associated with the presented techniques that prompt possible future research directions are discussed. Originality/value Despite considerable progress in the recent years, no comprehensive review on the analytical and computational research techniques to predict aircraft post-stall/ spin characteristics has been undertaken in the recent years.

show abstract

Reinforcement Learning-Based Optimal Flat Spin Recovery for Unmanned Aerial Vehicle

Cited by 30 publications

References 17 publications

Deep Reinforcement Learning for Integrated Non-Linear Control of Autonomous UAVs

Deep Reinforcement Learning for Integrated Non-Linear Control of Autonomous UAVs

Communication Emitter Motion Behavior’s Cognition Based on Deep Reinforcement Learning

A review and historical development of analytical techniques to predict aircraft spin and recovery characteristics

Contact Info

Product

Resources

About