Neural networks in virtual reference tuning

Abstract

This paper discusses the application of the Virtual Reference Tuning (VRT) techniques to tune neural controllers from experimental input-ouput data, by particularising nonlinear VRT and suitably computing gradients backpropagating in time. The flexibility of gradient computation with neural networks also allows alternative block diagrams with extra inputs to be considered. The neural approach to VRT in a closed loop setup is compared to the linear VRFT one in a simulated crane example.

Data-Driven Model-Free Tracking Reinforcement Learning Control with VRFT-based Adaptive Actor-Critic

“…Using this fact, it follows from (10) that the ORM tracking errors s( An NN can be used as a controller for nonlinear state-feedback control learning. Nonlinear VRFT is proposed in [41,42] and successfully applied to NN controllers in [41,[43][44][45] but only for output feedback control and not for state-feedback control as in here.…”

“…including the virtual states of the ORM), the process' initial states would suffice for this purpose. However, state extension is required for preserving the Markov property of the system (3) in order to ensure the correct collection of the transition samples; this is not possible otherwise without special collection design such as using a zero-order-hold for two-by-two consecutive time samples [43]. Correct transition samples collection is required for adaptive actor-critic tuning approach of the same NN controller that is initially tuned via VRFT.…”

Virtual Reference Feedback Tuning of MIMO Data-Driven Model-Free Adaptive Control Algorithms

“…for the tracking error, ) (k u in (7) can be considered to emerge from the I/O nonlinear recurrent controller )), such that the reference model output and the closed-loop CS output have similar trajectories. By abuse of notation, )) The two o.f.s in (9) and (10) can be made approximately equal for a rich parameterization of the controller which can be, for example, a neural network [10], [11]. Our mixed data-driven control approach is based on considering that ) (k r specific to VRFT equals ) 1 ( *  k y specific to MFAC:…”

Data-driven controller design for general MIMO nonlinear systems via virtual reference feedback tuning and neural networks

“…In [44], Chi et al presented a unified data-driven design framework of optimality-based generalized iterative learning control. VRFT was also applied to nonlinear systems by neural controllers [45] and MIMO linear systems [46,37].…”