The production process of a smart factory is complex and dynamic. As the core of manufacturing management, the research into the flexible job shop scheduling problem (FJSP) focuses on optimizing scheduling decisions in real time, according to the changes in the production environment. In this paper, deep reinforcement learning (DRL) is proposed to solve the dynamic FJSP (DFJSP) with random job arrival, with the goal of minimizing penalties for earliness and tardiness. A double deep Q-networks (DDQN) architecture is proposed and state features, actions and rewards are designed. A soft ε-greedy behavior policy is designed according to the scale of the problem. The experimental results show that the proposed DRL is better than other reinforcement learning (RL) algorithms, heuristics and metaheuristics in terms of solution quality and generalization. In addition, the soft ε-greedy strategy reasonably balances exploration and exploitation, thereby improving the learning efficiency of the scheduling agent. The DRL method is adaptive to the dynamic changes of the production environment in a flexible job shop, which contributes to the establishment of a flexible scheduling system with self-learning, real-time optimization and intelligent decision-making.

In various circumstances, such as human-robot interactions and industrial processes, planning at trajectory level is very useful to produce better movement. In this paper we present a near time optimal approach to plan a trajectory joining via points in real time for robot manipulators. To limit the speed variation, the path is smoothed around via-points in a limited area. The concept of online trajectory generation enables systems to react instantaneously at control level to unforeseen events. Simulation and real-world experimental results carried out on a KUKA Light-Weight Robot arm are presented.

The problems of CNC machine tool (CNCMT) fault diagnosis and production rescheduling have attracted continuous attention because of their great significance to the manufacturing industry. Digital twin is a supporting technology for achieving smart manufacturing and provides a new paradigm for solving these problems. This paper explores a digital twin-driven interaction and cooperation framework and proposes the architecture and implementation mechanism to enable the sharing of data, knowledge, and resource, to realize the fusion of physical space and cyber space, and to improve the accuracy of fault diagnosis. Under this framework, aiming at the influence of CNCMT failure on the initial production planning, a self-adaptation rescheduling method based on Monte Carlo Tree Search (MCTS) algorithm is proposed to provide support for developing more efficient production planning. Finally, the effectiveness of the proposed framework is validated by experimental study. The framework and integrated rescheduling approach can provide guidance for enterprises in implementing CNCMT maintenance and production scheduling to meet high accuracy and reliability requirements.

With the development of intelligent manufacturing, machine tools are considered the “mothership” of the equipment manufacturing industry, and the associated processing workshops are becoming more high-end, flexible, intelligent, and green. As the core of manufacturing management in a smart shop floor, research into the multi-objective dynamic flexible job shop scheduling problem (MODFJSP) focuses on optimizing scheduling decisions in real time according to changes in the production environment. In this paper, hierarchical reinforcement learning (HRL) is proposed to solve the MODFJSP considering random job arrival, with a focus on achieving the two practical goals of minimizing penalties for earliness and tardiness and reducing total machine load. A two-layer hierarchical architecture is proposed, namely the combination of a double deep Q-network (DDQN) and a dueling DDQN (DDDQN), and state features, actions, and external and internal rewards are designed. Meanwhile, a personal computer-based interaction feature is designed to integrate subjective decision information into the real-time optimization of HRL to obtain a satisfactory compromise. In addition, the proposed HRL framework is applied to multi-objective real-time flexible scheduling in a smart gear production workshop, and the experimental results show that the proposed HRL algorithm outperforms other reinforcement learning (RL) algorithms, metaheuristics, and heuristics in terms of solution quality and generalization and has the added benefit of real-time characteristics.

Handing an object over to a human is a challenging task for a robot to perform, especially when the human partner has no experience interacting with robots. This paper presents our work to enable a robot to learn how to achieve this task with wrist force/torque sensing. Firstly, we present a device to record the data, then we discuss the techniques used for the teaching. We choose to focus on the classification problem defined to enable the robot to find the finger opening movement. The main challenge is that the classification should be run online, at a comparable rate to the controller. To achieve a computationally efficient classifier, the Wavelet Packet Transformation is used for feature extraction, and then we used the Fisher criterion to reduce the dimension of features. A Relevance Vector Machine is used for the continuous classification procedure mainly for its sparsity. Some recorded data and the results from dimension reduction are shown, then we discuss the accuracy and sparsity of the classification by Relevance Vector Machine in this application. The software of continuous classification on forces is then tested on the robot for interactive object exchange between human and robot, which gives promising results.

Abstract. In this paper, we present a new solution to build a reactive trajectory controller for object manipulation in Human Robot Interaction (HRI) context. Using an online trajectory generator, the controller build a time-optimal trajectory from the actual state to a target situation every control cycle. A human aware motion planner provides a trajectory for the robot to follow or a point to reach. The main functions of the controller are its capacity to track a target, to follow a trajectory with respect to a task frame, or to switch to a new trajectory each time the motion planner provides a new trajectory. The controller chooses a strategy from different control modes depending on the situation. Visual servoing by trajectory generation and control is presented as one application of the approache. To illustrate the potential of the approach, some manipulation results are presented.

No abstract

No abstract