How to train your robot with deep reinforcement learning: lessons we have learned

Abstract

Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.

High-Tech Defense Industries: Developing Autonomous Intelligent Systems

“…On the ot the human intelligence literature usually defines intelligence as the human abilit over time and adapt to the external environment [24,25]. Moreover, the artific gence literature refers that intelligence is the capacity of mimicking human in [26], such as the ability of knowledge and reasoning [27], problem-solving [28], c cating, interacting, and learning [29]. Huang and Rust [26] and Huang et al [30] posed three artificial intelligences-mechanical, thinking, and feeling (Figure 1) Mechanical artificial intelligence is used for simple, standardized, repetitive tine tasks.…”

Cooperative Object Transportation Using Curriculum-Based Deep Reinforcement Learning

“…To overcome this problem, deep reinforcement learning (DRL) has received much attention in the machine learning field as an alternative to conventional RLs [ 22 , 23 , 24 ]. The most popular DRL application is Atari 2600 games with Google DeepMind [ 22 ]; the action-value function (Q-function) is approximated by a deep convolutional neural network called the deep Q-network (DQN).…”

The Societal Implications of Deep Reinforcement Learning

“…Though there are still considerable barriers to large-scale real-world deployment of DRL systems, we are beginning to see potential for application in areas including robotics (Ibarz et al, 2021), online personalisation and targeting (Zhao et al, 2019), finance (Fischer, 2018), autonomous driving (Tai et al, 2016), healthcare (Esteva et al, 2019), and data centre cooling (Gasparik et al, 2018). These applications and others will likely become more widespread as the technology improves.…”