Interactive Q‐Learning Approach for Pick‐and‐Place Optimization of the Die Attach Process in the Semiconductor Industry

Abstract

In semiconductor back-end production, the die attach process is one of the most critical steps affecting overall productivity. Optimization of this process can be modeled as a pick-and-place problem known to be NP-hard. Typical approaches are rule-based and metaheuristic methods. The two have high or low generalization ability, low or high performance, and short or long search time, respectively. The motivation of this paper is to develop a novel method involving only the strengths of these methods, i.e., high generalization ability and performance and short search time. We develop an interactive Q-learning in which two agents, a pick agent and a place agent, are trained and find a pick-and-place (PAP) path interactively. From experiments, we verified that the proposed approach finds a shorter path than the genetic algorithm given in previous research.

Rural road ecological landscape planning system based on interactive genetic algorithm

“…The MCU selected in the hardware adopts dual 12 bit A/D band sampling and holding internal reference source, and has dual 12 bit A/D synchronous conversion capability. It controls different working modes of the system hardware through SPI interface, and uploads and saves the buffered data [5] . According to the data acquisition instructions issued by the single chip microcomputer, the instrument collects the whole building structure, vegetation coverage, path and other related data of the vegetation landscape site through the data acquisition module.…”

Analysis and design of algorithms for the manufacturing process of integrated circuits

“…The described process is known as semiconductor chip placement process and can be formulated as a typical pick-and-place problem, which is NP-hard [9]. This paper analyzes the mathematical formulation of the problem and the (approximate) solution proposals presented in two previous papers [8,10]. These methods include eight heuristic rules that determine the movement of the robotic arm, a genetic algorithm that uses encoding based on binary matrices as chromosomes [8], and an interactive Q-learning algorithm with two agents: one for collection and the other for placement [10].…”

“…This paper analyzes the mathematical formulation of the problem and the (approximate) solution proposals presented in two previous papers [8,10]. These methods include eight heuristic rules that determine the movement of the robotic arm, a genetic algorithm that uses encoding based on binary matrices as chromosomes [8], and an interactive Q-learning algorithm with two agents: one for collection and the other for placement [10]. In our analysis, we identified the limitations of the mathematical formulation in [8] and proposed a new one that obtains an optimal solution for a given instance.…”

Review of Learning-Based Robotic Manipulation in Cluttered Environments

“…Robotic manipulation can be performed in different circumstances for different purposes that have been addressed in different studies [ 15 ]. For example, deep RL approaches have been used to assist robots in performing sophisticated robotic manipulation tasks in various applications, such as deformable object manipulation [ 16 ], heavy object manipulation [ 17 ], and pick-to-place tasks [ 18 , 19 , 20 ]. Even though several studies have focused on the learning approach (e.g., deep RL) to solve robotic manipulation problems, it still requires further studies, as stated in [ 21 ].…”