Reinforcement Learning for Extended Reality: Designing Self-Play Scenarios

Abstract

A common problem for deep reinforcement learning networks is a lack of training data to learn specific tasks through generalization. In this paper, we discuss using extended reality to train reinforcement learning agents to overcome this problem. We review popular reinforcement learning and extended reality techniques and then synthesize the information, this allowed us to develop our proposed design for a self learning agent. Meta learning offers an important way forward, but the agents ability to perform self-play is considered crucial for achieving successful AI. Therefore, we focus on improving self-play scenarios for teaching self-learning agents, by providing a supportive environment for improved agent-environment interaction.

Indoor Scene Recognition via Object Detection and TF-IDF

“…In the field of embodied agents and reinforcement learning, the improved speed of YOLO could be beneficial when performing studies investigating reinforcement learning and object navigation. This is because reinforcement learning is already computationally expensive, and using a relatively “simple” object detection framework could be beneficial [ 34 , 35 ]. Additionally, we are aware of the current controversy revolving around YOLOv4 [ 36 ] and YOLOv5 [ 32 ] and have no reason to select one over the other.…”

Indoor Scene Recognition via Object Detection and TF-IDF

“…We trained a recent implementation of YOLO [22,34], YOLOv5L This is because reinforcement learning is already computationally expensive, and using 173 a relatively "simple" object detection framework could be beneficial [35,36]. Additionally,…”