Adaptive Laser Welding Control: A Reinforcement Learning Approach

Abstract

Despite extensive research efforts in the field of laser welding, the imperfect repeatability of the weld quality still represents an open topic. Indeed, the inherent complexity of the underlying physical phenomena prevents the implementation of an effective controller using conventional regulators. To close this gap, we propose the application of Reinforcement Learning for closed-loop adaptive control of welding processes. The presented system is able to autonomously learn a control law that achieves a predefined weld quality independently from the starting conditions and without prior knowledge of the process dynamics. Specifically, our control unit influences the welding process by modulating the laser power and uses optical and acoustic emission signals as sensory input. The algorithm consists of three elements: a smart agent interacting with the process, a feedback network for quality monitoring, and an encoder that retains only the quality critic events from the sensory input. Based on the data representation provided by the encoder, the smart agent decides the output laser power accordingly. The corresponding input signals are then analyzed by the feedback network to determine the resulting process quality. Depending on the distance to the targeted quality, a reward is given to the agent. The latter is designed to learn from its experience by taking the actions that maximize not just its immediate reward, but the sum of all the rewards that it will receive from that moment on. Two learning schemes were tested for the agent, namely Q-Learning and Policy Gradient. The required training time to reach the targeted quality was 20 min for the former technique and 33 min for the latter.

Machine learning monitoring for laser osteotomy

“…Another perspective is to develop a complete self‐learning framework for tissues and laser regime classification. Finally, an extension of an adaptive control loop based on self‐learning that is able to adapt in real‐time the laser energy dose depending on the desired results is planned [33].…”

Deep reinforcement learning in production systems: a systematic literature review

“…The reward design can be based on different target variables, such as real-time profits (Powell, Machalek, and Quah 2020), cost-per-time function (Quah, Machalek, and Powell 2020), or similarity measures based on specified performance criteria (He et al 2020). The individual goal-oriented design enables a broad application in further applications such as flotation processes to reduce non-dynamic drawbacks of modelbased approaches (Jiang et al 2018), in laser welding to increase process repeatabilities (Masinelli et al (2020), and others), or in injection molding to broaden up narrow process windows of conventional methods in ultrahigh precision processes . A detailed list of all process control applications and related publications can be found in Table 3.…”

Improving Build Quality in Laser Powder Bed Fusion Using High Dynamic Range Imaging and Model-Based Reinforcement Learning

“…In the domain of laser welding, the authors of [56] used RL to control the laser power based on optical and acoustic measurement signals. Two different RL approaches (i.e., Qlearning and policy gradient algorithms) were investigated.…”