Accelerating Deep Continuous Reinforcement Learning through Task Simplification

Kerzel, Matthias; Mohammadi, Hadi Beik; Zamani, Mohsen; Wermter, Stefan

doi:10.1109/ijcnn.2018.8489712

Cited by 13 publications

(18 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The approaches used in the Kerzel et al [15] and Fournier et al [16] studies have similarities with the approach proposed in this study, including employing a start-from-simple strategy to accelerate the learning process, which can be categorized as curriculum learning. However, they use two-joint planar arm for a position reach environment, which is difficult to adjust into a more complicated environment.…”

Section: Related Workmentioning

confidence: 90%

“…By changing the precision requirement ( ) up to every epoch, we add a new task to the curriculum, which can smooth the training process and is easy to implement. Compared to the similar methods show in the Kerzel et al [15] and Fournier et al [16] studies, our continuous curriculum strategy obviates intensive computation for learning status evaluation and can be utilized in a real robotic arm.…”

Section: Related Workmentioning

confidence: 97%

“…This approach of learning from a fatal experience can be viewed as a curriculum that trains the easier-met goals first. Kerzel et al [15] accelerated reach-for-grasp learning by dynamically adjusting the target size. When the robot has little knowledge of the task, the Kerzel et al approach increases the target size to make it easier to succeed and get positive rewards for learning the policy.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Accelerating Reinforcement Learning for Reaching Using Continuous Curriculum Learning

Luo

Kasaei

Schomaker

2020

2020 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Reinforcement learning has shown great promise in the training of robot behavior due to the sequential decision making characteristics. However, the required enormous amount of interactive and informative training data provides the major stumbling block for progress. In this study, we focus on accelerating reinforcement learning (RL) training and improving the performance of multi-goal reaching tasks. Specifically, we propose a precision-based continuous curriculum learning (PCCL) method in which the requirements are gradually adjusted during the training process, instead of fixing the parameter in a static schedule. To this end, we explore various continuous curriculum strategies for controlling a training process. This approach is tested using a Universal Robot 5e in both simulation and realworld multi-goal reach experiments. Experimental results support the hypothesis that a static training schedule is suboptimal, and using an appropriate decay function for curriculum learning provides superior results in a faster way.

show abstract

Section: Related Workmentioning

confidence: 90%

Section: Related Workmentioning

confidence: 97%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Accelerating Reinforcement Learning for Reaching Using Continuous Curriculum Learning

Luo

Kasaei

Schomaker

2020

2020 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…The process is known for developmental psychology to solve the problem of collecting the enormous amount of required training samples in a realistic time that surpasses the possibilities of many robotic platforms [21]. Unity 3D machine learning-based research on robot arm control includes DRL strategies to train the robotic arm through machine learning, using the reward function for intelligent control of the robotic arm [22].…”

Section: Grasp Taskmentioning

confidence: 99%

Vision-Based Robotic Arm Control Algorithm Using Deep Reinforcement Learning for Autonomous Objects Grasping

et al. 2021

View full text Add to dashboard Cite

While working side-by-side, humans and robots complete each other nowadays, and we may say that they work hand in hand. This study aims to evolve the grasping task by reaching the intended object based on deep reinforcement learning. Thereby, in this paper, we propose a deep deterministic policy gradient approach that can be applied to a numerous-degrees-of-freedom robotic arm towards autonomous objects grasping according to their classification and a given task. In this study, this approach is realized by a five-degrees-of-freedom robotic arm that reaches the targeted object using the inverse kinematics method. You Only Look Once v5 is employed for object detection, and backward projection is used to detect the three-dimensional position of the target. After computing the angles of the joints at the detected position by inverse kinematics, the robot’s arm is moved towards the target object’s emplacement thanks to the algorithm. Our approach provides a neural inverse kinematics solution that increases overall performance, and its simulation results reveal its advantages compared to the traditional one. The robot’s end grip joint can reach the targeted location by calculating the angle of every joint with an acceptable range of error. However, the accuracy of the angle and the posture are satisfied. Experiments reveal the performance of our proposal compared to the state-of-the-art approaches in vision-based grasp tasks. This is a new approach to grasp an object by referring to inverse kinematics. This method is not only easier than the standard one but is also more meaningful for multi-degrees of freedom robots.

show abstract

“…Fournier et al [19] and Kerzel et al [20] also present approaches to simplify goals. However, instead of masking the goals, they simplify tasks by reducing the required precision for successful completion.…”

Section: A Curriculum Learning For Deep Reinforcement Learningmentioning

confidence: 99%

Curriculum goal masking for continuous deep reinforcement learning

Eppe

Magg

Wermter

2019

2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)

Self Cite

View full text Add to dashboard Cite

Deep reinforcement learning has recently gained a focus on problems where policy or value functions are independent of goals. Evidence exists that the sampling of goals has a strong effect on the learning performance, but there is a lack of general mechanisms that focus on optimizing the goal sampling process. In this work, we introduce goal masking as a method to estimate a goal's difficulty level and to exploit this estimation to realize curriculum learning. Our results indicate that focusing on goals with a medium difficulty level is appropriate for deep deterministic policy gradient (DDPG) methods, while an "aim for the stars and reach the moon-strategy", where difficult goals are sampled much more often than simple goals, leads to the best learning performance in cases where DDPG is combined with hindsight experience replay (HER).

show abstract

Accelerating Deep Continuous Reinforcement Learning through Task Simplification

Cited by 13 publications

References 13 publications

Accelerating Reinforcement Learning for Reaching Using Continuous Curriculum Learning

Accelerating Reinforcement Learning for Reaching Using Continuous Curriculum Learning

Vision-Based Robotic Arm Control Algorithm Using Deep Reinforcement Learning for Autonomous Objects Grasping

Curriculum goal masking for continuous deep reinforcement learning

Contact Info

Product

Resources

About