Reinforcement learning based control of batch polymerisation processes

Singh, Vikas; Kodamana, Hariprasad

doi:10.1016/j.ifacol.2020.06.111

Cited by 22 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Clustering problem: grouping of samples of tea according to their fermentation degree (HCA) [34] • Dimensionality reduction problem: dimension compression of process data to address high correlations between different variables and reduce the computational cost during the prediction of a polypropylene melt index using GP (PCA) [ The algorithm learns an optimal policy that selects which is the best action to execute given the state of the environment Control of polymerization processes [48,49] Dynamic programming Monte Carlo methods Temporal difference…”

Section: Supervised Learningmentioning

confidence: 99%

“…Reinforcement learning has recently gained increasing popularity in control tasks in process industries, in robotics and gaming since AlphaGo, a computer program, managed to defeat a professional Go player in 2015 [26,[48][49][50][51]. As such, its principal application spectrum in process engineering is related to process control problems.…”

Section: Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning in Chemical Product Engineering: The State of the Art and a Guide for Newcomers

2021

View full text Add to dashboard Cite

Chemical Product Engineering (CPE) is marked by numerous challenges, such as the complexity of the properties–structure–ingredients–process relationship of the different products and the necessity to discover and develop constantly and quickly new molecules and materials with tailor-made properties. In recent years, artificial intelligence (AI) and machine learning (ML) methods have gained increasing attention due to their performance in tackling particularly complex problems in various areas, such as computer vision and natural language processing. As such, they present a specific interest in addressing the complex challenges of CPE. This article provides an updated review of the state of the art regarding the implementation of ML techniques in different types of CPE problems with a particular focus on four specific domains, namely the design and discovery of new molecules and materials, the modeling of processes, the prediction of chemical reactions/retrosynthesis and the support for sensorial analysis. This review is further completed by general guidelines for the selection of an appropriate ML technique given the characteristics of each problem and by a critical discussion of several key issues associated with the development of ML modeling approaches. Accordingly, this paper may serve both the experienced researcher in the field as well as the newcomer.

show abstract

Section: Supervised Learningmentioning

confidence: 99%

Section: Reinforcement Learningmentioning

confidence: 99%

Machine Learning in Chemical Product Engineering: The State of the Art and a Guide for Newcomers

2021

View full text Add to dashboard Cite

show abstract

“…To overcome this, deep Q-learning has been introduced by Mnih et al, wherein the neural networks are used as function approximators for approximating the Q-values [18]. Some of the current works on RL applications in chemical processes employing Q-learning and deep Q-learning are related to applications such as polymerization and chromatography [19,20]. Another approach for solving an RL problem is known as policy gradients which directly optimizes a policy without using a value function or Q-value explicitly.…”

Section: Introductionmentioning

confidence: 99%

TASAC: a twin-actor reinforcement learning framework with stochastic policy for batch process control

Joshi¹,

Kodamanaa²,

Kandath³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Due to their complex nonlinear dynamics and batch-to-batch variability, batch processes pose a challenge for process control. Due to the absence of accurate models and resulting plant-model mismatch, these problems become harder to address for advanced model-based control strategies. Reinforcement Learning (RL), wherein an agent learns the policy by directly interacting with the environment, offers a potential alternative in this context. RL frameworks with actor-critic architecture have recently become popular for controlling systems where state and action spaces are continuous. It has been shown that an ensemble of actor and critic networks further helps the agent learn better policies due to the enhanced exploration due to simultaneous policy learning. To this end, the current study proposes a stochastic actor-critic RL algorithm, termed Twin Actor Soft Actor-Critic (TASAC), by incorporating an ensemble of actors for learning, in a maximum entropy framework, for batch process control.

show abstract

“…A good number of important results have been introduced by using ADP techniques to find approximate solutions to different problems, including Bellman himself, and most are related to the Markov decision processes . Only a few of the application results addressed the optimal control problem of constrained batch processes, see, for example. − This indicates that the issue of RL for batch processes with constraints remains open.…”

Section: Introductionmentioning

confidence: 99%

Data-Efficient Constrained Learning for Optimal Tracking of Batch Processes

Zhou

Gao

et al. 2021

Ind. Eng. Chem. Res.

View full text Add to dashboard Cite

Repeatability provides an opportunity to learn from historical process data, thus enabling batch processes to produce high-value and batch-improved products. However, industrial batch processes are always constrained. This makes many data-driven learning methods impractical. Therefore, learning within a set of constraints is of great importance in batch processes. In order to optimize tracking problems, iterative learning approaches require pre-established policies and knowledge of the underlying performance evaluation. In this article, we propose a constrained learning algorithm for the problem that is data-efficient, in the sense that it can be used by extracting useful information from historical process data under complex and uncertain conditions. Using historical process data, we create an efficient convex constraint for dynamic optimization problems, making the real-time optimal learning less dependent on model accuracy, which is the main tenet of model-based iterative learning. Our method achieves better control performance and a faster convergence rate than conventional methods. The technique is demonstrated using an injection molding process for tracking control tasks.

show abstract

Reinforcement learning based control of batch polymerisation processes

Cited by 22 publications

References 19 publications

Machine Learning in Chemical Product Engineering: The State of the Art and a Guide for Newcomers

Machine Learning in Chemical Product Engineering: The State of the Art and a Guide for Newcomers

TASAC: a twin-actor reinforcement learning framework with stochastic policy for batch process control

Data-Efficient Constrained Learning for Optimal Tracking of Batch Processes

Contact Info

Product

Resources

About