Reinforcement Learning-Based Path Generation Using Sequential Pattern Reduction and Self-Directed Curriculum Learning

Abstract

Recent advancements in robots and deep learning have led to active research in human-robot interaction. However, non-physical interaction using visual devices such as laser pointers has gained less attention than physical interaction using complex robots such as humanoids. Such vision-based interaction has high potential for use in recent human-robot collaboration environments such as assembly guidance, even with a minimum amount of configuration. In this paper, we introduce a simple robotic laser pointer device that follows an arbitrary planar path and is designed to be a visual instructional aid. We also propose an image-based automatic path generation method using reinforcement learning and a sequential pattern reduction technique. However, such vision-based human-robot interaction is generally performed in a dynamic environment, and it can frequently be necessary to calibrate the devices more than once. In this paper, we avoid the need for this re-calibration process through episodic randomization learning and improved learning efficiency. In particular, contrary to previous approaches, the agent controls the curriculum difficulty in a self-directed manner to determine the optimal curriculum. To our knowledge, this is the first study of curriculum learning that incorporates an explicit learning environment control signal initiated by the agent itself. Through quantitative and qualitative analyses, we show that the proposed selfdirected curriculum learning method outperforms ordinary episodic randomization and curriculum learning. We hope that the proposed method can be extended to a general reinforcement learning framework. INDEX TERMS Path generation, robotic laser pointer, deep reinforcement learning, curriculum learning I. INTRODUCTION A. BACKGROUND VOLUME 0

[Retracted] Construction Path of Academic English Learning Environment in Colleges and Universities from the Perspective of Education Ecology

“…Finally, relevant managers should also create a harmonious interpersonal academic environment within the scope of the university. By cultivating academic teams and research groups, students are encouraged to explore and study academic knowledge, which will help to promote the improvement of the academic level of the university and promote the long-term development of the university [ 3 , 4 ].…”

Curricula for teaching end-users to kinesthetically program collaborative robots

“…While prior work on curriculum-based learning for robotics has primarily been used to teach robots rather than humans (e.g., [27,28]), recent years have seen the application of curriculum-based learning to teach humans how to better use robots, particularly in the areas of robot-assisted surgery (e.g., [29,30]) and rehabilitation (e.g., [31]). In this work, we apply curriculum-based learning to kinesthetic teaching and investigate how it impacts users' programming perceptions and proficiency compared to practice-based learning representative of current training practices for kinesthetic teaching (Fig 1).…”

Controller Design of Tracking WMR System Based on Deep Reinforcement Learning

“…This study considers that the input space and output space of the motion control of the tracking unmanned vehicle are continuous, so in this study, the depth under the Actor-Critic (AC) framework suitable for continuous input and output space is adopted. Deterministic policy gradient (DPG) algorithm is the process framework of the AC algorithm shown in Figure 5 [22]. This study considers that the input space and output space of the motion control of the tracking unmanned vehicle are continuous, so in this study, the depth under the Actor-Critic (AC) framework suitable for continuous input and output space is adopted.…”

“…This study considers that the input space and output space of the motion control of the tracking unmanned vehicle are continuous, so in this study, the depth under the Actor-Critic (AC) framework suitable for continuous input and output space is adopted. Deterministic policy gradient (DPG) algorithm is the process framework of the AC algorithm shown in Figure 5 [22]. In the DPG algorithm, the samples in the training process are stored in the replay memory in turn, and a certain amount of mini batch samples are randomly selected for training.…”