MazeExplorer: A Customisable 3D Benchmark for Assessing Generalisation in Reinforcement Learning

Harries, Luke; Lee, Sebastian; Rzepecki, Jaroslaw; Hofmann, Katja; Devlin, Sam

doi:10.1109/cig.2019.8848048

Cited by 9 publications

(6 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the authors' opinion, the results obtained in this paper indicate that those techniques could be successfully utilized in order to solve non-trivial problems within such domain, given sufficient learning time. However, due to the high complexity of both of those algorithms and complete randomisation of the quest location generation (proven to impair the reinforcement learning process [66]), the presented solution is far from convergence. Applying this solution to a real-life scenario would yield improved results due to the partially predictable patterns of location distribution-resulting from population density, consumption patterns and more fixed pick-up points.…”

Section: Discussionmentioning

confidence: 99%

Deep Reinforcement Learning Algorithms for Path Planning Domain in Grid-like Environment

Grzelczak

Duch

2021

Applied Sciences

View full text Add to dashboard Cite

Recently, more and more solutions have utilised artificial intelligence approaches in order to enhance or optimise processes to achieve greater sustainability. One of the most pressing issues is the emissions caused by cars; in this paper, the problem of optimising the route of delivery cars is tackled. In this paper, the applicability of the deep reinforcement learning algorithms with regards to the aforementioned problem is tested on a simulation game designed and implemented to pose various challenges such as constant change of delivery locations. The algorithms chosen for this task are Advantage Actor-Critic (A2C) with and without Proximal Policy Optimisation (PPO). These novel and advanced reinforcement learning algorithms have yet not been utilised in similar scenarios. The differences in performance and learning process of those are visualised and discussed. It is demonstrated that both of those algorithms present a slow but steady learning curve, which is an expected effect of reinforcement learning algorithms, leading to a conclusion that the algorithms would discover an optimal policy with an adequately long learning process. Additionally, the benefits of the Proximal Policy Optimisation algorithm are proven by the enhanced learning curve with comparison to the Advantage Actor-Critic approach, as the learning process is characterised by faster growth with a significantly smaller variation. Finally, the applicability of such algorithms in the described scenarios is discussed, alongside the possible improvements and future work.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep Reinforcement Learning Algorithms for Path Planning Domain in Grid-like Environment

Grzelczak

Duch

2021

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…More recent work has produced game-like environments with procedurally generated elements, such as the Procgen Benchmark [18], MazeExplorer [30], and the Obstacle Tower environment [38]. However, we argue that, compared to NetHack or Minecraft, these environments do not provide the depth likely necessary to serve as long-term RL testbeds due to limited number of entities and environment interactions that agents have to learn to master.…”

Section: Related Workmentioning

confidence: 91%

The NetHack Learning Environment

Küttler,

Nardelli,

Miller

et al. 2020

Preprint

View full text Add to dashboard Cite

Progress in Reinforcement Learning (RL) algorithms goes hand-in-hand with the development of challenging environments that test the limits of current methods. While existing RL environments are either sufficiently complex or based on fast simulation, they are rarely both. Here, we present the NetHack Learning Environment (NLE), a scalable, procedurally generated, stochastic, rich, and challenging environment for RL research based on the popular single-player terminalbased roguelike game, NetHack. We argue that NetHack is sufficiently complex to drive long-term research on problems such as exploration, planning, skill acquisition, and language-conditioned RL, while dramatically reducing the computational resources required to gather a large amount of experience. We compare NLE and its task suite to existing alternatives, and discuss why it is an ideal medium for testing the robustness and systematic generalization of RL agents. We demonstrate empirical success for early stages of the game using a distributed Deep RL baseline and Random Network Distillation exploration, alongside qualitative analysis of various agents trained in the environment. NLE is open source at https://github.com/facebookresearch/nle.Preprint. Under review.

show abstract

“…Experimental setup The experiments are conducted to evaluate the success ratio, sample-efficiency, and generalization of our method, as well as compare our work to competitive baseline methods in multi-target environments. We develop and conduct experiments on (1) visual navigation tasks based on ViZDoom [23,18], and (2) robot arm manipulation tasks based on MuJoCo [42]. These multitarget tasks involve N targets which are placed at positions p 1 , p 2 , • • • , p N , and provide the index of one target as the goal z ∈ {1, 2, • • • , N }.…”

Section: Methodsmentioning

confidence: 99%

“…Reinforcement learning (RL) has been expanding to various fields including robotics, to solve increasingly complex problems. For instance, RL has been gradually mastering skills such as robot arm/hand manipulation on an object [2,48,36,22] and navigation to a target destination [18,39]. However, to benefit humans like the R2-D2 robot in the Star Wars, RL must extend to realistic settings that require interaction with multiple objects or destinations, which is still challenging for RL.…”

Section: Introductionmentioning

confidence: 99%

Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learning

Kim¹,

Lee²,

Kim³

et al. 2021

Preprint

View full text Add to dashboard Cite

Learning in a multi-target environment without prior knowledge about the targets requires a large amount of samples and makes generalization difficult. To solve this problem, it is important to be able to discriminate targets through semantic understanding. In this paper, we propose goal-aware cross-entropy (GACE) loss, that can be utilized in a self-supervised way using auto-labeled goal states alongside reinforcement learning. Based on the loss, we then devise goal-discriminative attention networks (GDAN) which utilize the goal-relevant information to focus on the given instruction. We evaluate the proposed methods on visual navigation and robot arm manipulation tasks with multi-target environments and show that GDAN outperforms the state-of-the-art methods in terms of task success ratio, sample efficiency, and generalization. Additionally, qualitative analyses demonstrate that our proposed method can help the agent become aware of and focus on the given instruction clearly, promoting goal-directed behavior.

show abstract

MazeExplorer: A Customisable 3D Benchmark for Assessing Generalisation in Reinforcement Learning

Cited by 9 publications

References 4 publications

Deep Reinforcement Learning Algorithms for Path Planning Domain in Grid-like Environment

Deep Reinforcement Learning Algorithms for Path Planning Domain in Grid-like Environment

The NetHack Learning Environment

Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learning

Contact Info

Product

Resources

About