Control architecture for embedding reinforcement learning frameworks on industrial control hardware

Schmidt, Alexander; Schellroth, Florian; Riedel, Oliver

doi:10.1145/3378184.3378198

Cited by 6 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, the decisions are automatically communicated ondemand between the two devices: The control unit providing the real-time capable system to fulfill the industrial requirements and a computer providing the ML framework to enable iterative learning. Instead of exchanging decisions, a message broker communication between the two software systems can be established to exchange the whole model to be executed on the controller (Schmidt et al, 2020). Furthermore, simple RL algorithms using discrete state-action-space and their framework can also be implemented directly within the control code (Demirkıran et al, 2020;Hameed & Schwung, 2020).…”

Section: Data Driven Process Optimization In Industrial Controlmentioning

confidence: 99%

Safe contextual Bayesian optimization integrated in industrial control for self-learning machines

et al. 2023

View full text Add to dashboard Cite

Intelligent manufacturing applications and agent-based implementations are scientifically investigated due to the enormous potential of industrial process optimization. The most widespread data-driven approach is the use of experimental history under test conditions for training, followed by execution of the trained model. Since factors, such as tool wear, affect the process, the experimental history has to be compiled extensively. In addition, individual machine noise implies that the models are not easily transferable to other (theoretically identical) machines. In contrast, a continual learning system should have the capacity to adapt (slightly) to a changing environment, e.g., another machine under different working conditions. Since this adaptation can potentially have a negative impact on process quality, especially in industry, safe optimization methods are required. In this article, we present a significant step towards self-optimizing machines in industry, by introducing a novel method for efficient safe contextual optimization and continuously trading-off between exploration and exploitation. Furthermore, an appropriate data discard strategy and local approximation techniques enable continual optimization. The approach is implemented as generic software module for an industrial edge control device. We apply this module to a steel straightening machine as an example, enabling it to adapt safely to changing environments.

show abstract

Section: Data Driven Process Optimization In Industrial Controlmentioning

confidence: 99%

Safe contextual Bayesian optimization integrated in industrial control for self-learning machines

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Further categories with single publications are listed in Table 7. These include specific topics such as building an agent swapping framework to allow learning in a nonreal-time environment and execution in a real-time environment (Schmidt, Schellroth, and Riedel 2020) or the deep RL based selection of optimal prediction models in the semiconductor manufacturing domain to cope with demand fluctuations and avoid shortages and overstock (Chien, Lin, and Lin 2020).…”

Section: Further Applicationsmentioning

confidence: 99%

Deep reinforcement learning in production systems: a systematic literature review

Panzer

Bender

2021

International Journal of Production Research

103

View full text Add to dashboard Cite

Shortening product development cycles and fully customisable products pose major challenges for production systems. These not only have to cope with an increased product diversity but also enable high throughputs and provide a high adaptability and robustness to process variations and unforeseen incidents. To overcome these challenges, deep Reinforcement Learning (RL) has been increasingly applied for the optimisation of production systems. Unlike other machine learning methods, deep RL operates on recently collected sensor-data in direct interaction with its environment and enables real-time responses to system changes. Although deep RL is already being deployed in production systems, a systematic review of the results has not yet been established. The main contribution of this paper is to provide researchers and practitioners an overview of applications and to motivate further implementations and research of deep RL supported production systems. Findings reveal that deep RL is applied in a variety of production domains, contributing to data-driven and flexible processes. In most applications, conventional methods were outperformed and implementation efforts or dependence on human experience were reduced. Nevertheless, future research must focus more on transferring the findings to real-world systems to analyse safety aspects and demonstrate reliability under prevailing conditions.

show abstract

Mapping of Standardized State Machines to Utilize Machine Learning Models in Process Control Environments

Grothoff

Kleinert

2021

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Control architecture for embedding reinforcement learning frameworks on industrial control hardware

Cited by 6 publications

References 14 publications

Safe contextual Bayesian optimization integrated in industrial control for self-learning machines

Safe contextual Bayesian optimization integrated in industrial control for self-learning machines

Deep reinforcement learning in production systems: a systematic literature review

Mapping of Standardized State Machines to Utilize Machine Learning Models in Process Control Environments

Contact Info

Product

Resources

About