Simulation-as-a-Service for Reinforcement Learning Applications by Example of Heavy Plate Rolling Processes

Scheiderer, Christian; Thun, Timo; Idzik, Christian; Posada-Moreno, Andrés Felipe; Krämer, A.; Lohmar, Johannes; Meisen, Tobias

doi:10.1016/j.promfg.2020.10.126

Cited by 12 publications

(11 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, Gamal et al [18] demonstrated that RL in combination with process data identifies model parameters and thus improve model predictions for bar and wire hot rolling processes. For hot rolling, Scheiderer et al [1] published a RL approach, which can design pass schedules considering several optimization objectives. The authors used a database of simulation data to train the RL algorithm.…”

Section: Reinforcement Learning For Process Optimizationmentioning

confidence: 99%

“…Therefore, process design is often only based on experience of long-time rolling mill operators or fast process models. Previous work by Scheiderer et al [1] showed that coupling RL with a FRM is very promising. RL is applied, since after a successful training process it is capable of solving given problems, e.g.…”

Section: Introductionmentioning

confidence: 99%

“…This manual design approach does not lead to an optimization accounting for multiple objectives. Previous work [1] has shown the potential of coupling reinforcement learning (RL) with fast analytical rolling models (FRM) to optimize hot rolling processes. However, the designed pass schedules do not robustly reach the desired final height within typical industrial tolerances.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Advanced automatic pass schedule design for hot rolling by coupling reinforcement learning with a fast rolling model

Idzik

2023

Materials Research Proceedings

View full text Add to dashboard Cite

Abstract. Rolling is a well-established forming process for producing finished or semi-finished products in various industries. Although highly automated, most rolling processes are designed manually by experts based on their knowledge, highly specialized heuristics and analytical process models or numerical simulations. This manual design approach does not lead to an optimization accounting for multiple objectives. Previous work [1] has shown the potential of coupling reinforcement learning (RL) with fast analytical rolling models (FRM) to optimize hot rolling processes. However, the designed pass schedules do not robustly reach the desired final height within typical industrial tolerances. Therefore, in this paper the existing approach of coupling RL with an FRM is extended by dynamically ranges for height reductions. This extension guarantees that the target height is always reached exactly. In addition to the height reduction, the RL algorithm can determine the inter-pass time, initial slab temperature and rolling velocity. For the optimization, an objective function, called reward function, considering all relevant optimization objectives such as the final grain size and energy consumption, was developed. An exemplary training was performed for a defined starting (140 mm) and final height (25 mm). The resulting, automatically designed pass schedules reach the target height and fulfill all defined optimization objective including the required average austenite grain size.

show abstract

Section: Reinforcement Learning For Process Optimizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Advanced automatic pass schedule design for hot rolling by coupling reinforcement learning with a fast rolling model

Idzik

2023

Materials Research Proceedings

View full text Add to dashboard Cite

show abstract

“…By considering aerodynamic and machining parameters, an optimised path planning for the machine tool was developed, which reduces development time and cost. Among the publications listed in Table 6, other design approaches included 2D-strip packing to improve space utilisation in Zhu and Xiang Dong Li (2020) which reduced average gaps by 20% compared to several benchmark algorithms, or the design of a SaaS architecture in Scheiderer et al (2020) which significantly reduced optimisation times in heavy-plate rolling compared to manual tuning.…”

Section: (Process) Designmentioning

confidence: 99%

Deep reinforcement learning in production systems: a systematic literature review

Panzer

Bender

2021

International Journal of Production Research

104

View full text Add to dashboard Cite

Shortening product development cycles and fully customisable products pose major challenges for production systems. These not only have to cope with an increased product diversity but also enable high throughputs and provide a high adaptability and robustness to process variations and unforeseen incidents. To overcome these challenges, deep Reinforcement Learning (RL) has been increasingly applied for the optimisation of production systems. Unlike other machine learning methods, deep RL operates on recently collected sensor-data in direct interaction with its environment and enables real-time responses to system changes. Although deep RL is already being deployed in production systems, a systematic review of the results has not yet been established. The main contribution of this paper is to provide researchers and practitioners an overview of applications and to motivate further implementations and research of deep RL supported production systems. Findings reveal that deep RL is applied in a variety of production domains, contributing to data-driven and flexible processes. In most applications, conventional methods were outperformed and implementation efforts or dependence on human experience were reduced. Nevertheless, future research must focus more on transferring the findings to real-world systems to analyse safety aspects and demonstrate reliability under prevailing conditions.

show abstract

“…We highlight recent RL approaches developed for process planning and pass schedules design in heavy plate rolling processes [ 16 , 17 ] and open die forging processes [ 18 ]. Recently, Liu et al [ 19 ] have applied an RL method to free-from sheet-metal stamping.…”

Section: Introductionmentioning

confidence: 99%

A Reinforcement Learning Control in Hot Stamping for Cycle Time Optimization

Nievas

Pagès‐Bernaus

Bonada³

et al. 2022

Materials

View full text Add to dashboard Cite

Hot stamping is a hot metal forming technology increasingly in demand that produces ultra-high strength parts with complex shapes. A major concern in these systems is how to shorten production times to improve production Key Performance Indicators. In this work, we present a Reinforcement Learning approach that can obtain an optimal behavior strategy for dynamically managing the cycle time in hot stamping to optimize manufacturing production while maintaining the quality of the final product. Results are compared with the business-as-usual cycle time control approach and the optimal solution obtained by the execution of a dynamic programming algorithm. Reinforcement Learning control outperforms the business-as-usual behavior by reducing the cycle time and the total batch time in non-stable temperature phases.

show abstract

Simulation-as-a-Service for Reinforcement Learning Applications by Example of Heavy Plate Rolling Processes

Cited by 12 publications

References 15 publications

Advanced automatic pass schedule design for hot rolling by coupling reinforcement learning with a fast rolling model

Advanced automatic pass schedule design for hot rolling by coupling reinforcement learning with a fast rolling model

Deep reinforcement learning in production systems: a systematic literature review

A Reinforcement Learning Control in Hot Stamping for Cycle Time Optimization

Contact Info

Product

Resources

About