Reinforcement learning for industrial process control: A case study in flatness control in steel industry

Deng, Jifei; Sierla, Seppo; Sun, Jie; Vyatkin, Valeriy

doi:10.1016/j.compind.2022.103748

Cited by 40 publications

(6 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several works [5,[16][17][18] have compared conventional controllers with data-driven and adaptive DRL controllers or decision-support systems, noting that while DRL can solve non-linear complex and stochastic control problems, it has limitations such as lack of optimality guarantees and data requirements. Furthermore, in process or safety-critical industries, it is not recommended to completely replace the conventional controller and rely on such a learned policy that could fail in cases of unobserved states.…”

Section: Related Work and Contributionsmentioning

confidence: 99%

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces

Abbas,

Chasparis,

Kelleher

2024

Preprint

View full text Add to dashboard Cite

Traditional controllers have limitations as they rely on prior knowledge about the physics of the problem, require modeling of dynamics, and struggle to adapt to abnormal situations. Deep reinforcement learning has the potential to address these problems by learning optimal control policies through exploration in an environment. For safety-critical environments, it is impractical to explore randomly, and replacing conventional controllers with black-box models is also undesirable. Also, it is expensive in continuous state and action spaces, unless the search space is constrained. To address these challenges we propose a specialized deep residual policy safe reinforcement learning with a cycle of learning approach adapted for complex and continuous state-action spaces. Residual policy learning allows learning a hybrid control architecture where the reinforcement learning agent acts in synchronous collaboration with the conventional controller. The cycle of learning initiates the policy through the expert trajectory and guides the exploration around it. Further, the specialization through the input-output hidden Markov model helps to optimize policy that lies within the region of interest (such as abnormality), where the reinforcement learning agent is required and is activated. The proposed solution is validated on the Tennessee Eastman process control.

show abstract

Section: Related Work and Contributionsmentioning

confidence: 99%

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces

Abbas,

Chasparis,

Kelleher

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…By analyzing sensor data during the printing process, researchers can develop adaptive control strategies that modify process parameters on the fly in response to process variations or detected defects [49]. Reinforcement learning (RL) algorithms, such as Q-learning, Deep Q-Networks, and deep reinforcement learning, have been successfully employed in developing adaptive process control strategies for LPBF [50,51].…”

Section: Ai-supported Optimization Techniquesmentioning

confidence: 99%

Optimization potentials of laser powder bed fusion: A conceptual approach

Strutz,

Samardžić,

Šimunović

2023

FME Transactions

View full text Add to dashboard Cite

Additive manufacturing (AM), more specifically laser powder bed fusion (LPBF), has become increasingly important for the production of complex components. Despite recent improvements, issues with process parameter optimization, multi-material approaches, CAx chain, adaption for automated mass production, automated process planning, and quality control are still major concerns. So far, despite growing interest, the technology has not yet made the leap into everyday and large-scale use. The use of artificial intelligence offers opportunities to solve many of these problems and improve LPBF technology. In this paper, these topics are addressed to give the reader a holistic overview of the potential for optimization. The individual topics are not only explained and supported with example products from various industries but also evaluated in terms of cost-effectiveness and quality improvement. By evaluating the potentials, restrictions, and recommendations, a framework is created for further investigation and practical application of optimization approaches.

show abstract

“…Moreover, this test demonstrates the controller's applicability because it was only trained for one of the parts. Other PPO applications can be found in other manufacturing processes, such as controlling the power and velocity of a laser in charge of melting via powder bed fusion [64] and controlling the rolls of a strip rolling process to achieve the desired flatness [144]. It should be noted that this last application is also compared with DRL hybrid algorithms, outperforming them regarding results and stability.…”

Section: Process Controlmentioning

confidence: 99%

A Review of Deep Reinforcement Learning Approaches for Smart Manufacturing in Industry 4.0 and 5.0 Framework

et al. 2022

View full text Add to dashboard Cite

In this review, the industry’s current issues regarding intelligent manufacture are presented. This work presents the status and the potential for the I4.0 and I5.0’s revolutionary technologies. AI and, in particular, the DRL algorithms, which are a perfect response to the unpredictability and volatility of modern demand, are studied in detail. Through the introduction of RL concepts and the development of those with ANNs towards DRL, the potential and variety of these kinds of algorithms are highlighted. Moreover, because these algorithms are data based, their modification to meet the requirements of industry operations is also included. In addition, this review covers the inclusion of new concepts, such as digital twins, in response to an absent environment model and how it can improve the performance and application of DRL algorithms even more. This work highlights that DRL applicability is demonstrated across all manufacturing industry operations, outperforming conventional methodologies and, most notably, enhancing the manufacturing process’s resilience and adaptability. It is stated that there is still considerable work to be carried out in both academia and industry to fully leverage the promise of these disruptive tools, begin their deployment in industry, and take a step closer to the I5.0 industrial revolution.

show abstract

Reinforcement learning for industrial process control: A case study in flatness control in steel industry

Cited by 40 publications

References 33 publications

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces

Optimization potentials of laser powder bed fusion: A conceptual approach

A Review of Deep Reinforcement Learning Approaches for Smart Manufacturing in Industry 4.0 and 5.0 Framework

Contact Info

Product

Resources

About