Setting up a Reinforcement Learning Task with a Real-World Robot

Mahmood, A. Rupam; Korenkevych, Dmytro; Komer, Brent; Bergstra, James

doi:10.48550/arxiv.1803.07067

Cited by 2 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The variation in performance between different runs was small and did not diverge over time except on Create-Mover, where the sequences of experience became dissimilar over time across runs. These results are a testament to the tight control over system delays achieved in our tasks by using the computational model of Mahmood et al (2018).…”

Section: Resultsmentioning

confidence: 75%

“…The control interface offers low-level position and velocity control commands. We use UR5 to develop two tasks called UR-Reacher-2 and UR-Reacher-6 based on the tasks developed by Mahmood et al (2018).…”

Section: Robotsmentioning

confidence: 99%

“…We use the Reacher task with UR5 developed by Mahmood et al (2018), which is designed analogously to OpenAI-Gym Reacher (Brockman et al 2016). We modify the reward function and call the task UR-Reacher-2.…”

Section: Ur-reacher-2mentioning

confidence: 99%

“…All of these tasks are implemented following the computational model for real-time reinforcement learning tasks described by Mahmood et al (2018). We improve on that model by running environment and agent computations on two different processes, which we found to reduce execution delays compared to the use of Python threads.…”

Section: Create-movermentioning

confidence: 99%

“…Due to the lack of benchmark tasks, it is hard to analyze these difficulties and address them as a community. Mahmood et al (2018) recently brought to attention some of the difficulties of real-world robot learning and showed that learning performance can be highly sensitive to different elements of the task setup such as the action space, the action cycle time defined by the time between two subsequent actions, and system delays (also see Riedmiller 2012). Therefore, reproducing and utilizing existing results can be hard when the details of these task setup elements are omitted.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Setting up a Reinforcement Learning Task with a Real-World Robot

Mahmood¹,

Korenkevych²,

Komer³

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Through many recent successes in simulation, model-free reinforcement learning has emerged as a promising approach to solving continuous control robotic tasks. The research community is now able to reproduce, analyze and build quickly on these results due to open source implementations of learning algorithms and simulated benchmark tasks. To carry forward these successes to real-world applications, it is crucial to withhold utilizing the unique advantages of simulations that do not transfer to the real world and experiment directly with physical robots. However, reinforcement learning research with physical robots faces substantial resistance due to the lack of benchmark tasks and supporting source code. In this work, we introduce several reinforcement learning tasks with multiple commercially available robots that present varying levels of learning difficulty, setup, and repeatability. On these tasks, we test the learning performance of off-the-shelf implementations of four reinforcement learning algorithms and analyze sensitivity to their hyper-parameters to determine their readiness for applications in various real-world tasks. Our results show that with a careful setup of the task interface and computations, some of these implementations can be readily applicable to physical robots. We find that state-of-the-art learning algorithms are highly sensitive to their hyper-parameters and their relative ordering does not transfer across tasks, indicating the necessity of re-tuning them for each task for best performance. On the other hand, the best hyper-parameter configuration from one task may often result in effective learning on held-out tasks even with different robots, providing a reasonable default. We make the benchmark tasks publicly available to enhance reproducibility in real-world reinforcement learning 1 .

show abstract

Section: Resultsmentioning

confidence: 75%

Section: Robotsmentioning

confidence: 99%

Section: Ur-reacher-2mentioning

confidence: 99%

Section: Create-movermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Setting up a Reinforcement Learning Task with a Real-World Robot

Mahmood¹,

Korenkevych²,

Komer³

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

show abstract

Cascaded Fuzzy Reward Mechanisms in Deep Reinforcement Learning for Comprehensive Path Planning in Textile Robotic Systems

Zhao,

Ding,

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

With the rapid advancement of industrial automation and artificial intelligence technologies, particularly in the textile industry, robotic technology is increasingly challenged with intelligent path planning and executing high-precision tasks. This study focuses on the automatic path planning and yarn-spool-assembly tasks of textile robotic arms, proposing an end-to-end planning and control model that integrates deep reinforcement learning. The innovation of this paper lies in the introduction of a cascaded fuzzy reward system, which is integrated into the end-to-end model to enhance learning efficiency and reduce ineffective exploration, thereby accelerating the convergence of the model. A series of experiments conducted in a simulated environment demonstrate the model’s exceptional performance in yarn-spool-assembly tasks. Compared to traditional reinforcement learning methods, our model shows potential advantages in improving task success rates and reducing collision rates. The cascaded fuzzy reward system, a core component of our end-to-end deep reinforcement learning model, offers a novel and more robust solution for the automated path planning of robotic arms. In summary, the method proposed in this study provides a new perspective and potential applications for industrial automation, especially in the operation of robotic arms in complex and uncertain environments.

show abstract

Setting up a Reinforcement Learning Task with a Real-World Robot

Cited by 2 publications

References 0 publications

Setting up a Reinforcement Learning Task with a Real-World Robot

Setting up a Reinforcement Learning Task with a Real-World Robot

Cascaded Fuzzy Reward Mechanisms in Deep Reinforcement Learning for Comprehensive Path Planning in Textile Robotic Systems

Contact Info

Product

Resources

About