Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review

Abstract

Deep learning has provided new ways of manipulating, processing and analyzing data. It sometimes may achieve results comparable to, or surpassing human expert performance, and has become a source of inspiration in the era of artificial intelligence. Another subfield of machine learning named reinforcement learning, tries to find an optimal behavior strategy through interactions with the environment. Combining deep learning and reinforcement learning permits resolving critical issues relative to the dimensionality and scalability of data in tasks with sparse reward signals, such as robotic manipulation and control tasks, that neither method permits resolving when applied on its own. In this paper, we present recent significant progress of deep reinforcement learning algorithms, which try to tackle the problems for the application in the domain of robotic manipulation control, such as sample efficiency and generalization. Despite these continuous improvements, currently, the challenges of learning robust and versatile manipulation skills for robots with deep reinforcement learning are still far from being resolved for real-world applications.

Deep Learning Control for Digital Feedback Systems: Improved Performance with Robustness against Parameter Change

“…The number of neurons in each of the hidden layers is taken as five. For two hidden layers, this number is almost in conformity with the empirical rule of Equation (13). The nonlinear activation function for all hidden nodes is the ReLU function which has the merit of solving the vanishing gradient problem for deep NNs trained by the BP algorithm [6,7].…”

“…RL thereby achieves long-term results, which are otherwise very difficult to achieve. Deep RL has recently been used in robotic manipulation controllers [13,14]. A deep learning controller based on RL is also implemented in [15] for the application of DL in industrial process control.…”

Reinforcement Learning for Pick and Place Operations in Robotics: A Survey

“…A broad-scale generalized strategy for implementing pick-and-place with RL in robotics is needed before this technique can be implemented at scale. Mohammed and Chua [12], Liu et al [13], and Tai et al [14] have all written review papers focused on RL in robotics; however, these papers have a broad range of focus in terms of robotic agents used and the task completed.…”

Sample Efficient Deep Reinforcement Learning via Uncertainty Estimation

“…This makes applying DRL in the real world challenging, for example in robotics (Sünderhauf et al, 2018;Dulac-Arnold et al, 2019). In tasks like manipulation, sample collection is a slow and costly process (Liu et al, 2021). It is even more expensive in riskaverse applications like autonomous driving (Kothari et al, 2021).…”