Design and evaluation of advanced intelligent flight controllers

Abstract

Reinforcement learning based methods could be feasible of solving adaptive optimal control problems for nonlinear dynamical systems. This work presents a proof of concept for applying reinforcement learning based methods to robust and adaptive flight control tasks. A framework for designing and examining these methods is introduced by means of the open research civil aircraft model (RCAM) and optimality criteria. A state-of-the-art robust flight controller -the incremental nonlinear dynamic inversion (INDI) controller -serves as a reference controller. Two intelligent control methods are introduced and examined. The deep deterministic policy gradient (DDPG) controller is selected as a promising actor critic reinforcement learning method that currently gains much attraction in the field of robotics. In addition, an adaptive version of a proportional-integral-derivative (PID) controller, the PID neural network (PIDNN) controller, is selected as the second method. The results show that all controllers are able to control the aircraft model. Moreover, the PIDNN controller exhibits improved reference tracking if a good initial guess of its weights is available. In turn, the DDPG algorithm is able to control the nonlinear aircraft model while minimizing a multi-objective value function. This work provides insight into the usability of selected intelligent controllers as flight control functions as well as a comparison to state-of-the-art flight control functions.

Dynamic Inversion-Based Control Concept for Transformational Tilt-Wing eVTOLs

“…Sensory NDI [10,38,39], but incremental NDI as well, replace the dependency on the internal dynamics 𝑓 (𝑥) through the sensor measurements ˆ 𝑥 and û, yielding the implicit control law…”

Flight Testing Total Energy Control Autopilot Functionalities for High Altitude Aircraft

“…This baseline controller is a rate-command, attitude-hold controller based on gain-scheduled proportional and integral feedback, see [3] for details. For the flight experiments on the Cessna Citation, however, an INDI inner loop controller [8,12] and an LPV controller [7] are available. Both have the same input-output interface as the HALE baseline controller, i.e., receiving pitch angle commands from the autopilot and generating the elevator deflection angle 𝛿 𝑒 .…”

Two-Loop Acceleration Autopilot Design and Analysis Based on TD3 Strategy

“…In order to reduce the time and workload for identifying model information during the autopilot design process, literature [24] developed a learning-based design method for UAV autopilot using the DDPG algorithm by designing appropriate observation and reward functions. In addition, literature [25] made a comparative analysis of PID neural network controller and DDPG controller and also provided a conceptual proof that reinforcement learning can effectively solve the adaptive optimal control problem of nonlinear dynamic systems. However, most of the methods in [19,20,22,24] and [25] were designed and analyzed based on nonglobal profile state rather than entire flight envelope, which led to insufficient consideration of global constraints and performance indicators.…”

“…In addition, literature [25] made a comparative analysis of PID neural network controller and DDPG controller and also provided a conceptual proof that reinforcement learning can effectively solve the adaptive optimal control problem of nonlinear dynamic systems. However, most of the methods in [19,20,22,24] and [25] were designed and analyzed based on nonglobal profile state rather than entire flight envelope, which led to insufficient consideration of global constraints and performance indicators. When there was great uncertainty in the flight environment, the autonomy and robustness of the nominal trajectory tracking guidance mode were poor.…”

“…(1) Contrasting to the previous alternatives [19,20,22,24,25] based on nonglobal profile state, herein the proposed method takes the entire flight envelope state space as the research object. The TD3 algorithm is used to offline learn the control parameters for the entire flight envelope, which can ensure the autopilot has good performance during the flight envelope.…”