Fault-Tolerant Control of Programmable Logic Controller-Based Production Systems With Deep Reinforcement Learning

Abstract

Fault-tolerant control policies that automatically restart PLC-based aPS during fault recovery can increase system availability. This paper provides a proof-of-concept that such policies can be synthesized with DRL. The authors specifically focus on systems with multiple end-effectors that are actuated in only one or two axes, commonly used for assembly and logistics tasks. Due to the large number of actuators in multi-end-effector systems and the limited possibilities to track workpieces in a single coordinate system, these systems are especially challenging to learn. This paper demonstrates that a hierarchical multi-agent DRL approach together with a separate coordinate prediction module per agent can overcome these challenges. The evaluation of the suggested approach on the simulation of a small laboratory demonstrator shows that it is capable of restarting the system and completing open tasks as part of fault recovery.

Aplicación de algoritmos de aprendizaje automático a sistemas robóticos multiagente para la programación y control de operaciones productivas y logísticas: una revisión de la literatura reciente

“…Además, los procesos industriales que van involucrando gran número de MARS que cooperan entre sí, utilizan variedad de tecnologías como DL, DRL, QL y algoritmos de aprendizaje para alcanzar objetivos comunes. A continuación, la Tabla 2 muestra la agrupación de los algoritmos de aprendizaje, ciertas características particulares y finalmente ejemplos aplicados a producción y control industrial de operaciones productivas y logísticas (Zinn et al, 2021).…”

“…Adicional, la Tabla 3, explica las categorías, los tipos de algoritmo, la definición y ejemplos que en la revisión bibliográfica se pudo encontrar (Zinn et al, 2021).…”

A review of the applications of multi-agent reinforcement learning in smart factories

“…Due to the non-synchronicity of the state transitions between both machines, an asynchronous updating rule is also incorporated in the learning process. Zinn et al (2021) presented a MARL system based on DQN and actor–critic to learn the distributed fault-tolerant control policies for automated production systems during fault recovery to increase availability. Liu et al (2022) proposed a multi-agent DQN approach to make maintenance scheduling decisions for personnel and also production control during maintenance.…”

Control System Development and Implementation of a CNC Laser Engraver for Environmental Use with Remote Imaging

“…The introduction of modern progressive technologies in all spheres of human activity was presented in the concept of the fourth industrial revolution (Industry 4.0) [ 1 – 5 ]; resulting in making it possible to apply and integrate modern information technologies such as Artificial Intelligence (AI) [ 6 , 7 ], Big Data (BD) [ 8 , 9 ], and Internet of Things (IoT) with hardware solutions based on Microcontroller Unit (MCU) [ 10 – 14 ]. Accordingly, it becomes possible to automate the production process through the introduction of new control systems such as Computer Numerical Control (CNC) machines or Programmable Logic Controller (PLC) [ 15 – 17 ]. The implementation of CNC and PLC provides a wide opportunity to optimize the production process, improve the adaptation flexibility and customization for the release of new products, and minimize the influence of the human factor at the production stage.…”