An Open-Source Multi-goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet

Yang, Xintong; Ji, Ze; Wu, Jing; Lai, Yu‐Kun

doi:10.1007/978-3-030-89177-0_2

Cited by 10 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To investigate the effect of the proposed A 2 method, we conduct a series of simulation experiments using the Mini-Grid [14] and PMG environments [15]. All performances displayed were the success rates of achieving the final goal without demonstrations, averaged over five random seeds.…”

Section: Resultsmentioning

confidence: 99%

“…Manipulation tasks: We use the ChestPush and ChestPick tasks with one block and the BlockStack task with two blocks from the Pybullet Multigoal (PMG) environments [15]. The state representation and reward function remain the same as the original paper.…”

Section: Resultsmentioning

confidence: 99%

“…In this paper, we highlight the effectiveness of their integration into DRL algorithms for learning multi-step, sparse reward tasks. We conducted a series of ablation studies on three representative off-policy DRL agents, i.e., DQN (deterministic, discrete actions) [11], DDPG (deterministic, continuous actions) [12] and SAC (stochastic, continuous actions) [13], using the Mini-grid [14] and Pybullet Multigoal environments [15].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Abstract Demonstrations and Adaptive Exploration for Efficient and Stable Multi-step Sparse Reward Reinforcement Learning

Yang¹,

Ji²,

Wu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Although Deep Reinforcement Learning (DRL) has been popular in many disciplines including robotics, state-of-theart DRL algorithms still struggle to learn long-horizon, multistep and sparse reward tasks, such as stacking several blocks given only a task-completion reward signal. To improve learning efficiency for such tasks, this paper proposes a DRL exploration technique, termed A 2 , which integrates two components inspired by human experiences: Abstract demonstrations and Adaptive exploration. A 2 starts by decomposing a complex task into subtasks, and then provides the correct orders of subtasks to learn. During training, the agent explores the environment adaptively, acting more deterministically for well-mastered subtasks and more stochastically for ill-learnt subtasks. Ablation and comparative experiments are conducted on several grid-world tasks and three robotic manipulation tasks. We demonstrate that A 2 can aid popular DRL algorithms (DQN, DDPG, and SAC) to learn more efficiently and stably in these environments.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Abstract Demonstrations and Adaptive Exploration for Efficient and Stable Multi-step Sparse Reward Reinforcement Learning

Yang¹,

Ji²,

Wu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Tasks and the GMDP To examine the effectiveness of the proposed methods, three robotic manipulation tasks developed in simulation are used in the experiments, including the ChestPush, the ChestPick and the BlockStack tasks from the Pybullet Multigoal (PMG) simulation software [8], as shown in Fig. 2.…”

Section: Methodsmentioning

confidence: 99%

Accelerating Multi-step Sparse Reward Reinforcement Learning

Yang,

2024

Proceedings of the Cardiff University Engineering Research Conference 2023

View full text Add to dashboard Cite

After the great successes of deep reinforcement learning (DRL) in recent years, developing methods to speed up DRL algorithms for more complex tasks closer to those in the real world has become increasingly important. In particular, there is a lack of research on long-horizon tasks that contain multiple subtasks or intermediate steps and can only provide sparse rewards at task completion point. This paper suggests to 1) use human priors to decompose a task and provide abstract demonstrations – the correct sequences of steps to guide exploration and learning, and 2) adjust the exploration parameters adaptively according to the online performances of the policy. The proposed ideas are implemented on three popular DRL algorithms, and experimental results on gridworld and manipulation tasks prove the concept and effectiveness of the proposed techniques.

show abstract

“…PyBullet is utilized for simulation of the actual robot motions [17,18] . This environment is widely utilized as a robot learning environment for manipulation due to its portability and light weight for variety of machine learning tasks [19]. After the program is created in the simulation environment, it is applied to control the actual robot.…”

Section: Introductionmentioning

confidence: 99%

Control Program Generator for Vehicle Robot using Grammatical Evolution

Sukarman,

Sato,

Kita

2023

Adv. sci. technol. eng. syst. j.

View full text Add to dashboard Cite

A robot development has spread widely for various purposes. It is difficult to create a control program for an autonomous mobile robot manually. Therefore, an automatic design of the control program for an autonomous mobile robot is proposed in this research. The autonomous mobile robot is created with LEGO MINDSTORMS EV3, and the control program for the autonomous mobile robot is designed using Grammatical Evolution (GE). Grammatical Evolution (GE), which is one of the evolutionary computations, is designed to generate a program or a program fragment satisfying the design objective. PyBullet is used with GE to simulate the behavior of the robot. A robot traveling along a trajectory was considered as an example. GE can generate the control program of the robot behavior of a robot vehicle traveling along a trajectory. The computer simulation reveals the robot can travel along a designated line. Since there is a reality gap between the simulator and the real environment, the parameters of the vehicle robot such as produced power and sensor sensitivity are calibrated to reduce the gap. Comparison of the computer simulation and the experimental result shows that the reproducibility of the vehicle trajectory in the real environment is high.

show abstract

An Open-Source Multi-goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet

Cited by 10 publications

References 5 publications

Abstract Demonstrations and Adaptive Exploration for Efficient and Stable Multi-step Sparse Reward Reinforcement Learning

Abstract Demonstrations and Adaptive Exploration for Efficient and Stable Multi-step Sparse Reward Reinforcement Learning

Accelerating Multi-step Sparse Reward Reinforcement Learning

Control Program Generator for Vehicle Robot using Grammatical Evolution

Contact Info

Product

Resources

About