Virtual State Feedback Reference Tuning and Value Iteration Reinforcement Learning for Unknown Observable Systems Control

Rădac, Mircea-Bogdan; Borlea, Anamaria-Ioana

doi:10.3390/en14041006

Cited by 20 publications

(25 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both 𝒔 and 𝝆 will be replaced by their offline calculated counterparts 𝒔 and 𝝆 following the VSFRT principle. Problem ( 5) is indirectly solved as the next equivalent controller identification problem [3,4]…”

Section: The Model Reference Controlmentioning

confidence: 99%

See 1 more Smart Citation

Trajectory Tracking within a Hierarchical Primitive-Based Learning Approach

Rădac

2022

Entropy

Self Cite

View full text Add to dashboard Cite

A hierarchical learning control framework (HLF) has been validated on two affordable control laboratories: an active temperature control system (ATCS) and an electrical rheostatic braking system (EBS). The proposed HLF is data-driven and model-free, while being applicable on general control tracking tasks which are omnipresent. At the lowermost level, L1, virtual state-feedback control is learned from input–output data, using a recently proposed virtual state-feedback reference tuning (VSFRT) principle. L1 ensures a linear reference model tracking (or matching) and thus, indirect closed-loop control system (CLCS) linearization. On top of L1, an experiment-driven model-free iterative learning control (EDMFILC) is then applied for learning reference input–controlled outputs pairs, coined as primitives. The primitives’ signals at the L2 level encode the CLCS dynamics, which are not explicitly used in the learning phase. Data reusability is applied to derive monotonic and safely guaranteed learning convergence. The learning primitives in the L2 level are finally used in the uppermost and final L3 level, where a decomposition/recomposition operation enables prediction of the optimal reference input assuring optimal tracking of a previously unseen trajectory, without relearning by repetitions, as it was in level L2. Hence, the HLF enables control systems to generalize their tracking behavior to new scenarios by extrapolating their current knowledge base. The proposed HLF framework endows the CLCSs with learning, memorization and generalization features which are specific to intelligent organisms. This may be considered as an advancement towards intelligent, generalizable and adaptive control systems.

show abstract

Section: The Model Reference Controlmentioning

confidence: 99%

“…where 𝝅 is the controller function parameter leading to notation 𝒞(𝒔 , 𝝅) (here, the controller can be an NN or other type of approximator) [4,5]. In [4,5], it was motivated why the reference model state 𝒔 should not be included within 𝒔 because the former correlates with 𝝆 .…”

Section: The Model Reference Controlmentioning

confidence: 99%

Trajectory Tracking within a Hierarchical Primitive-Based Learning Approach

Rădac

2022

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…NN-based RL is different from supervised learning, which is a learning method that obtains training information from the environment. In recent years, it has made great progress in the field of intelligent control by combining RL with traditional control methods for reference tracking [25]- [27]. In [25], a novel virtual state-feedback tuning and value iteration RL were applied for learning linear reference model output tracking control of observable systems with unknown dynamics.…”

Section: Introductionmentioning

confidence: 99%

“…In recent years, it has made great progress in the field of intelligent control by combining RL with traditional control methods for reference tracking [25]- [27]. In [25], a novel virtual state-feedback tuning and value iteration RL were applied for learning linear reference model output tracking control of observable systems with unknown dynamics. In [26], the author uses RL method to improve the tracking accuracy and robustness of the H 2 control.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning-Based Fixed-Time Trajectory Tracking Control for Uncertain Robotic Manipulators With Input Saturation

Cao

Sun

Jiang

et al. 2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

A fixed-time trajectory tracking control method for uncertain robotic manipulators with input saturation based on reinforcement learning (RL) is studied. The designed reinforcement learning control algorithm is implemented by radial basis function (RBF) neural network, in which the actor neural network is used to generate the control strategy and the critic neural network is used to evaluate the execution cost. A new non-singular fast terminal sliding mode technique is used to ensure the convergence of tracking error in fixed time, and the upper bound of convergence time is estimated. To solve the saturation problem of an actuator, a nonlinear anti-windup compensator is designed to compensate for the saturation effect of the joint torque actuator in real time. Finally, the stability of the closed-loop system based on Lyapunov candidate is analyzed, and the timing convergence of the closed-loop system is proved. Simulation and experimental results show the effectiveness and superiority of the proposed control law.

show abstract

“…Recent RL development for nonlinear control systems has implications for aircraft guidance tasks. A Virtual State-feedback Reference Feedback Tuning (VSFRT) method [ 22 ] was applied to unknown observable systems control. In [ 23 ], a hierarchical soft actor–critic algorithm was proposed for task allocation which significantly improved the efficiency of the intelligent system.…”

Section: Introductionmentioning

confidence: 99%

A Multi-Dimensional Goal Aircraft Guidance Approach Based on Reinforcement Learning with a Reward Shaping Algorithm

Yang

Liu

et al. 2021

Sensors

View full text Add to dashboard Cite

Guiding an aircraft to 4D waypoints at a certain heading is a multi-dimensional goal aircraft guidance problem. [d=Zu]In order to improve the performance and solve this problem, this paper proposes a multi-layer RL approach.To enhance the performance, in the present study, a multi-layer RL approach to solve the multi-dimensional goal aircraft guidance problem is proposed. The approach [d=Zu]enablesassists the autopilot in an ATC simulator to guide an aircraft to 4D waypoints at certain latitude, longitude, altitude, heading, and arrival time, respectively. To be specific, a multi-layer RL [d=Zu]approach is proposedmethod to simplify the neural network structure and reduce the state dimensions. A shaped reward function that involves the potential function and Dubins path method is applied. [d=Zu]Experimental and simulation results show that the proposed approachExperiments are conducted and the simulation results reveal that the proposed method can significantly improve the convergence efficiency and trajectory performance. [d=Zu]FurthermoreFurther, the results indicate possible application prospects in team aircraft guidance tasks, since the aircraft can directly approach a goal without waiting in a specific pattern, thereby overcoming the problem of current ATC simulators.

show abstract

Virtual State Feedback Reference Tuning and Value Iteration Reinforcement Learning for Unknown Observable Systems Control

Cited by 20 publications

References 48 publications

Trajectory Tracking within a Hierarchical Primitive-Based Learning Approach

Trajectory Tracking within a Hierarchical Primitive-Based Learning Approach

Reinforcement Learning-Based Fixed-Time Trajectory Tracking Control for Uncertain Robotic Manipulators With Input Saturation

A Multi-Dimensional Goal Aircraft Guidance Approach Based on Reinforcement Learning with a Reward Shaping Algorithm

Contact Info

Product

Resources

About