Reinforcement Learning Based Production Control of Semi-automated Manufacturing Systems

Overbeck, Leonard; Hugues, Adrien; May, Marvin Carl; Kuhnle, Andreas; Lanza, Gisela

doi:10.1016/j.procir.2021.10.027

Cited by 14 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Anomaly detection [54] Root cause analysis of production disturbances or casual-relationship discovery [55] Production performance prediction [56], Human behavior control [57] Predictive scheduling [58], Adaptive production control [59] Figure 1. Four-Level and Four-Know categorization of ML applications.…”

Section: Systemmentioning

confidence: 99%

Machine Learning in Manufacturing towards Industry 4.0: From ‘For Now’ to ‘Four-Know’

et al. 2023

Self Cite

View full text Add to dashboard Cite

While attracting increasing research attention in science and technology, Machine Learning (ML) is playing a critical role in the digitalization of manufacturing operations towards Industry 4.0. Recently, ML has been applied in several fields of production engineering to solve a variety of tasks with different levels of complexity and performance. However, in spite of the enormous number of ML use cases, there is no guidance or standard for developing ML solutions from ideation to deployment. This paper aims to address this problem by proposing an ML application roadmap for the manufacturing industry based on the state-of-the-art published research on the topic. First, this paper presents two dimensions for formulating ML tasks, namely, ’Four-Know’ (Know-what, Know-why, Know-when, Know-how) and ’Four-Level’ (Product, Process, Machine, System). These are used to analyze ML development trends in manufacturing. Then, the paper provides an implementation pipeline starting from the very early stages of ML solution development and summarizes the available ML methods, including supervised learning methods, semi-supervised methods, unsupervised methods, and reinforcement methods, along with their typical applications. Finally, the paper discusses the current challenges during ML applications and provides an outline of possible directions for future developments.

show abstract

Section: Systemmentioning

confidence: 99%

Machine Learning in Manufacturing towards Industry 4.0: From ‘For Now’ to ‘Four-Know’

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…The battery production in European research institutions and industry is already, although slowly, experiencing a transition from the exclusively mass production of standard cells towards the additional operation of more agile production plants, which enable the profitable manufacturing of lower batch sizes and a more flexible choice of materials and cell shapes [11][12][13][14][15][16].…”

Section: Introductionmentioning

confidence: 99%

Methodology for the Holistic Design of Format-Flexible Lithium-Ion Battery Systems

Gandert,

Mühlpfort,

Müller-Welt

et al. 2024

International Journal of Energy Research

View full text Add to dashboard Cite

Most battery cells are developed according to a standard design and optimized regarding their electrical properties. However, firstly, there is a demand for an individual cell design adapted to the application. Secondly, additional properties, for example, thermal and mechanical, are decisive for the cell behavior and should be considered in the overall design. Therefore, the present work introduces a model-based development tool for the holistic design of format-flexible battery cells, which are optimized for their future application, allowing for different cell geometries within one battery system. The different approaches and simulation models for the single design steps are presented, and the prediction of the electrochemical, electrical, and thermal behaviors of the cell and whole battery system is described. A key aspect here is the appropriate interconnection of the single models in a way that helps to narrow down the multitude of interacting parameters in a systematic manner and, thus, makes a holistic design possible. In a last step, the functionality of the tool is shown in a case study by means of an e-bike battery pack.

show abstract

“…With a large market value [1], the manufacturing industry is enthusiastic to the ways that is conductive to the production efficiency. A number of studies have demonstrated that reinforcement learning may be applied to manufacturing processes and has the potential to dramatically improve productivity [2,3].…”

Section: Introductionmentioning

confidence: 99%

SMPL: Simulated Industrial Manufacturing and Process Control Learning Environments

Zhang¹,

Wang²,

Decardi‐Nelson³

et al. 2022

Preprint

View full text Add to dashboard Cite

Traditional biological and pharmaceutical manufacturing plants are controlled by human workers or pre-defined thresholds. Modernized factories have advanced process control algorithms such as model predictive control (MPC). However, there is little exploration of applying deep reinforcement learning to control manufacturing plants. One of the reasons is the lack of high fidelity simulations and standard APIs for benchmarking. To bridge this gap, we develop an easy-to-use library that includes five high-fidelity simulation environments: BeerFMTEnv, ReactorEnv, AtropineEnv, PenSimEnv and mAbEnv, which cover a wide range of manufacturing processes. We build these environments on published dynamics models. Furthermore, we benchmark online and offline, model-based and model-free reinforcement learning algorithms for comparisons of follow-up research. † † Official documentation:https://smpl-env.readthedocs.io/en/latest/index.html Official implementation: https://github.com/smpl-env/smpl Code of experiments: https://github.com/smpl-env/smpl-experiments Preprint. Under review.

show abstract

Reinforcement Learning Based Production Control of Semi-automated Manufacturing Systems

Cited by 14 publications

References 9 publications

Machine Learning in Manufacturing towards Industry 4.0: From ‘For Now’ to ‘Four-Know’

Machine Learning in Manufacturing towards Industry 4.0: From ‘For Now’ to ‘Four-Know’

Methodology for the Holistic Design of Format-Flexible Lithium-Ion Battery Systems

SMPL: Simulated Industrial Manufacturing and Process Control Learning Environments

Contact Info

Product

Resources

About