A model-based deep reinforcement learning method applied to finite-horizon optimal control of nonlinear control-affine system

Kim, Jong Woo; Park, Byung Jun; Yoo, Haeun; Oh, Tae Hoon; Lee, Jay H.; Lee, Jong Min

doi:10.1016/j.jprocont.2020.02.003

Cited by 46 publications

(11 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other approaches utilize metalearning and apprenticeship learning to quickly adapt trained RL models to new processes [27,28]. In the model-based setting, Kim et al [29] incorporate deep neural networks (DNNs) as value function approximators into the globalized dual heuristic programming algorithm. Predictive models have also been augmented with popular DRL algorithms, such as DDPG or TD3, to improve the policy gradient estimation [30] Other approaches to RL-based control postulate a fixed control structure such as PID [31,32,33].…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning

Lawrence,

Forbes,

Loewen

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep reinforcement learning (RL) is an optimization-driven framework for producing control strategies for general dynamical systems without explicit reliance on process models. Good results have been reported in simulation. Here we demonstrate the challenges in implementing a state of the art deep RL algorithm on a real physical system. Aspects include the interplay between software and existing hardware; experiment design and sample efficiency; training subject to input constraints; and interpretability of the algorithm and control law. At the core of our approach is the use of a PID controller as the trainable RL policy. In addition to its simplicity, this approach has several appealing features: No additional hardware needs to be added to the control system, since a PID controller can easily be implemented through a standard programmable logic controller; the control law can easily be initialized in a "safe" region of the parameter space; and the final product-a well-tuned PID controller-has a form that practitioners can reason about and deploy with confidence.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning

Lawrence,

Forbes,

Loewen

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Action refers to the combination of action elements according to a certain order to form an action sequence of a certain type of movement [4,5]. Activity refers to a broader concept, representing the complex movement of the human body, which is closely related to the object and the surrounding environment [6]. Recognition of human behavior is usually combined with multifeature fusion, and its application prospects are very broad.…”

Section: Introductionmentioning

confidence: 99%

Multifeature Fusion Human Motion Behavior Recognition Algorithm Using Deep Reinforcement Learning

2021

Mobile Information Systems

View full text Add to dashboard Cite

Through the recognition and analysis of human motion information, the actual motion state of human body can be obtained. However, the multifeature fusion of human behavior has limitations in recognition accuracy and robustness. Combined with deep reinforcement learning, multifeature fusion human behavior recognition is studied and we proposed a multifeature fusion human behavior recognition algorithm using deep reinforcement learning. Firstly, several typical human behavior data sets are selected as the research data in the benchmark data set. In the selected data sets, the behavior category contained in each video is the same behavior, and there are category tags. Secondly, the attention model is constructed. In the deep reinforcement learning network, the small sampling area is used as the model input. Finally, the corresponding position of the next visual area is estimated according to the time series information obtained after the input. The human behavior recognition algorithm based on deep reinforcement learning multifeature fusion is completed. The results show that the average accuracy of multifeature fusion of the algorithm is about 95%, the human behavior recognition effect is good, the identification accuracy rate is as high as about 98% and passed the camera movement impact performance test and the algorithm robustness, and the average time consumption of the algorithm is only 12.7 s, which shows that the algorithm has very broad application prospects.

show abstract

“…Recently, there has been growing interest amongst the research community and industry in the development of reinforcement learning (RL) based control schemes [1]. This is underpinned by the ability of RL to naturally account for process stochasticity and handle nonlinear dynamics, and reflected by a growing literature that demonstrates application empirically [2,3,4]. All of these works rely on offline simulation of a process model, with results often validated on the same model that the RL policy was trained.…”

Section: Introductionmentioning

confidence: 99%

Safe Chance Constrained Reinforcement Learning for Batch Process Control

Mowbray¹,

Petsagkourakis²,

Chanona³

et al. 2021

Preprint

View full text Add to dashboard Cite

Reinforcement Learning (RL) controllers have generated excitement within the control community. The primary advantage of RL controllers relative to existing methods is their ability to optimize uncertain systems independently of explicit assumption of process uncertainty. Recent focus on engineering applications has been directed towards the development of safe RL controllers. Previous works have proposed approaches to account for constraint satisfaction through constraint tightening from the domain of stochastic model predictive control. Here, we extend these approaches to account for plant-model mismatch. Specifically, we propose a data-driven approach that utilizes Gaussian processes for the offline simulation model and use the associated posterior uncertainty prediction to account for joint chance constraints and plant-model mismatch. The method is benchmarked against nonlinear model predictive control via case studies. The results demonstrate the ability of the methodology to account for process uncertainty, enabling satisfaction of joint chance constraints even in the presence of plant-model mismatch.

show abstract

A model-based deep reinforcement learning method applied to finite-horizon optimal control of nonlinear control-affine system

Cited by 46 publications

References 40 publications

Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning

Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning

Multifeature Fusion Human Motion Behavior Recognition Algorithm Using Deep Reinforcement Learning

Safe Chance Constrained Reinforcement Learning for Batch Process Control

Contact Info

Product

Resources

About