Integration of Discrete Simulation, Prediction, and Optimization Methods for a Production Line Digital Twin Design

Krenczyk, Damian; Paprocka, Iwona

doi:10.3390/ma16062339

Cited by 6 publications

(3 citation statements)

References 36 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 17 displays the production system design in which we compared the performance indicators of conventional (FIFO) and RL approaches for task distribution based on the model to optimize storage capacity. The production system was modeled using FlexSim simulation software for validation [30]. To implement RL, we used the PPO algorithm from the Stable-Baseline3 library [31], which is known for its effectiveness in solving numerous problems.…”

Section: Simulation Environmentmentioning

confidence: 99%

Factory Simulation of Optimization Techniques Based on Deep Reinforcement Learning for Storage Devices

Lim,

Jeong

2023

Applied Sciences

View full text Add to dashboard Cite

In this study, reinforcement learning (RL) was used in factory simulation to optimize storage devices for use in Industry 4.0 and digital twins. Industry 4.0 is increasing productivity and efficiency in manufacturing through automation, data exchange, and the integration of new technologies. Innovative technologies such as the Internet of Things (IoT), artificial intelligence (AI), and big data analytics are smartly automating manufacturing processes and integrating data with production systems to monitor and analyze production data in real time and optimize factory operations. A digital twin is a digital model of a physical product or process in the real world. It is built on data and real-time information collected through sensors and accurately simulates the behavior and performance of a real-world manufacturing floor. With a digital twin, one can leverage data at every stage of product design, development, manufacturing, and maintenance to predict, solve, and optimize problems. First, we defined an RL environment, modeled it, and validated its ability to simulate a real physical system. Subsequently, we introduced a method to calculate reward signals and apply them to the environment to ensure the alignment of the behavior of the RL agent with the task objective. Traditional approaches use simple reward functions to tune the behavior of reinforcement learning agents. These approaches issue rewards according to predefined rules and often use reward signals that are unrelated to the task goal. However, in this study, the reward signal calculation method was modified to consider the task goal and the characteristics of the physical system and calculate more realistic and meaningful rewards. This method reflects the complex interactions and constraints that occur during the optimization process of the storage device and generates more accurate episodes of reinforcement learning in agent behavior. Unlike the traditional simple reward function, this reflects the complexity and realism of the storage optimization task, making the reward more sophisticated and effective.The stocker simulation model was used to validate the effectiveness of RL. The model is a storage device that simulates logistics in a manufacturing production area. The results revealed that RL is a useful tool for automating and optimizing complex logistics systems, increasing the applicability of RL in logistics. We proposed a novel method for creating an agent through learning using the proximal policy optimization algorithm, and the agent was optimized by configuring various learning options. The application of reinforcement learning resulted in an effectiveness of 30–100%, and the methods can be expanded to other fields.

show abstract

Section: Simulation Environmentmentioning

confidence: 99%

Factory Simulation of Optimization Techniques Based on Deep Reinforcement Learning for Storage Devices

Lim,

Jeong

2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Digital twin technology can realize the interconnection and deep integration of the physical world and the virtual world. It possesses the capability to predict and control product states in the monitoring of complex product manufacturing processes [11].…”

Section: Fault Diagnosis Platform For Hot-rolled Strip Based On Digit...mentioning

confidence: 99%

Digital Twin-Based Fault Diagnosis Platform for Final Rolling Temperature in Hot Strip Production

Desheng,

Jian,

Mingxin

et al. 2023

Materials

View full text Add to dashboard Cite

The final rolling temperature in hot rolling is an important process parameter for hot-rolled strips and greatly influences their mechanical properties and rolling stability. The diagnosis of final rolling temperature anomalies in hot rolling has always been difficult in industry. A data-driven risk assessment method for detecting final rolling temperature anomalies is proposed. In view of the abnormal setting value for the strip head, a random forest model is established to screen the process parameters with high feature importance, and the isolation forest algorithm is used to evaluate the risk associated with the remaining parameters. In view of the abnormal process curve of the full length of the strip, the Hausdorff distance algorithm is used to eliminate samples with large deviations, and a risk assessment of the curve is carried out using the LCSS algorithm. Aiming to understand the complex coupling relationship between the influencing factors, a method for identifying the causes of anomalies, combining a knowledge graph and a Bayesian network, is established. According to the results of the strip head and the full-length risk assessment model, the occurrence of the corresponding nodes in the Bayesian network is determined, and the root cause of the abnormality is finally output. By combining mechanistic modeling and data modeling techniques, it becomes possible to rapidly, automatically, and accurately detect and analyze final rolling temperature anomalies during the rolling process. When applying the system in the field, when compared to manual analysis by onsite personnel, the accuracy of deducing the causes of anomalies was found to reach 92%.

show abstract

“…At a later stage, the model may be developed further, along with the subsequent phases of the production system life cycle, and refined to obtain a Digital Twin. Integration, including the exchange architecture, simulation creation, performance optimization, and predictive analysis of the production process conditions combine in order to obtain high flexibility of the production system [12].…”

Section: Introductionmentioning

confidence: 99%

Modeling and Simulation of a Digital Twin of a Production System for Industry 4.0 with Work-in-Process Synchronization

Kampa

2023

Applied Sciences

View full text Add to dashboard Cite

One of the main problems of modern manufacturing systems is the increasing complexity related to modern Industry 4.0 technologies that are fundamentally changing manufacturing and logistics processes and operations. Industry 4.0 includes, e.g., flexible automation and robotization, which make complex manufacturing systems difficult to analyze. Some modeling and simulation methods are being used to solve industrial problems and can serve as an interface between the production level and management level. The new trend of the Digital Twin, creating simulation models as similar as possible to the real system, and a Digital Twin framework for a manufacturing line from the automotive industry, was considered. Simulation models typically start from the empty state and some warmup time is required to achieve the stable state. The Key Performance Indicators were also analyzed for the stable state. However, there are many stochastic parameters such as machinery failures, human errors, quality issues, etc., that make the real processes differ from simulated processes, and cause the instability of production throughput and changes in the Work in Process. To analyze the Work in Process in the model, initialization of the model with proper production data is required, as the Digital Twin uses data synchronization with the production database. In this paper, the digital model of a human-robot-operated manufacturing system with Work-in-Process data synchronization is analyzed, and the results of the statistical analysis of simulation experiments are presented. The obtained results show high variability of finished production, which is related to system instability due to random failures, especially when the system starts from an empty state. However, an increase in initial Work in Process results in better efficiency and stability for the whole system. The DT simulation of the manufacturing system can be very helpful, as it becomes a repository of knowledge about the real system and enables the analysis of its dynamics. However, for proper functionality, the model should include information about the current WIP state, which enables the start of the simulation with exactly the same number of queues as in the real system. The presented method can also be used in similar enterprises from other industries, especially for those with discrete processes or high WIP variability, and for further synchronization of other DT parameters.

show abstract

Integration of Discrete Simulation, Prediction, and Optimization Methods for a Production Line Digital Twin Design

Cited by 6 publications

References 36 publications

Factory Simulation of Optimization Techniques Based on Deep Reinforcement Learning for Storage Devices

Factory Simulation of Optimization Techniques Based on Deep Reinforcement Learning for Storage Devices

Digital Twin-Based Fault Diagnosis Platform for Final Rolling Temperature in Hot Strip Production

Modeling and Simulation of a Digital Twin of a Production System for Industry 4.0 with Work-in-Process Synchronization

Contact Info

Product

Resources

About