A Data-Driven Model-Reference Adaptive Control Approach Based on Reinforcement Learning

Abstract

Model-reference adaptive systems refer to a consortium of techniques that guide plants to track desired reference trajectories. Approaches based on theories like Lyapunov, sliding surfaces, and backstepping are typically employed to advise adaptive control strategies. The resulting solutions are often challenged by the complexity of the reference model and those of the derived control strategies. Additionally, the explicit dependence of the control strategies on the process dynamics and reference dynamical models may contribute in degrading their efficiency in the face of uncertain or unknown dynamics. A model-reference adaptive solution is developed here for autonomous systems where it solves the Hamilton-Jacobi-Bellman equation of an error-based structure. The proposed approach describes the process with an integral temporal difference equation and solves it using an integral reinforcement learning mechanism. This is done in real-time without knowing or employing the dynamics of either the process or reference model in the control strategies. A class of aircraft is adopted to validate the proposed technique.

Modeling and control of boiler in thermal power plant using model reference adaptive control

“…The block diagram in Figure 2 shows the structure of a model reference adaptive control (MRAC) system that is composed of a process, controller, reference model, and adjustment mechanism block (Abouheaf et al 2021)).…”

Real-time measurement-driven reinforcement learning control approach for uncertain nonlinear systems

“…The work contributes an online measurement-driven adaptive learning mechanism that (i) adopts incremental learning capabilities to improve the control strategy in real-time, (ii) avoids incorporating the process and the reference-model dynamics explicitly into the underlying control strategy, (iii) provides a flexible feedback mechanism in terms of the order of the model-following dynamics, and (iv) allows for online approximate solutions for a class of optimal tracking problems. It builds on the contributions of [55] to develop an online IRL control mechanism for non-linear model-following systems with uncertain dynamics. The work presented herein supports the theoretical findings of [55] with solid data-processing and practical evidence.…”

“…It builds on the contributions of [55] to develop an online IRL control mechanism for non-linear model-following systems with uncertain dynamics. The work presented herein supports the theoretical findings of [55] with solid data-processing and practical evidence. A data-driven approach is developed for the real-time control of a 6-DoF Kinova robotic arm, as a highly nonlinear dynamic system.…”