Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisation

James, Stephen; Wada, Kazumi; Laidlow, Tristan; Davison, Andrew J.

doi:10.1109/cvpr52688.2022.01337

Cited by 35 publications

(42 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Akbulut et al [2] introduce a new framework called Adaptive Conditional Neural Movement Primitives, combining supervised learning and RL to conserve old skills learned from robot demonstrations while being adaptive to new environments. James and Davison [51] present a coarse-to-fine discrete RL algorithm to solve sparse reward manipulation tasks by using only a small amount of demonstration and exploration data (work extended by [49] and [50]). Celemin et al [19] include human corrective advice in the action domain through a learning-from-demonstration approach, while an RL algorithm guides the learning process by filtering out human feedback that does not maximize the reward.…”

Section: Learning From Demonstrationmentioning

confidence: 99%

“…Using Multiple Guided RL Methods First, we find that the guided RL-compliant papers tend to use a variety of guided RL methods. For instance, [7], [27], and [51] utilize at least three guided RL approaches, while [43], [70], and [82] deploy five or more approaches to obtain improvements in all three guided RL dimensions.…”

Section: Key Insights On Guided Rl Compliancementioning

confidence: 99%

See 1 more Smart Citation

Guided Reinforcement Learning: A Review and Evaluation for Efficient and Effective Real-World Robotics [Survey]

Eßer

Bach

Jestel

et al. 2023

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

ecent successes aside, reinforcement learning (RL) still faces significant challenges in its application to the real-world robotics domain. Guiding the learning process with additional knowledge offers a potential solution, thus leveraging the strengths of data-and knowledge-driven approaches. However, this field of research encompasses several disciplines and hence would benefit from a structured overview.In this article, we propose a concept of guided RL that provides a systematic approach toward accelerating the training process and improving performance for real-world robotics settings. We introduce a taxonomy that structures guided RL approaches and shows how different sources of knowledge can be integrated into the learning pipeline in a practical way. Based on this, we describe available approaches in this field and quantitatively evaluate their specific impact in terms of efficiency, effectiveness, and sim-to-real transfer within the robotics domain. However, learning control policies in such a way naturally requires many interactions with the environment. This emphasizes the importance of both collecting highquality samples and exploring the search space in a sample-efficient manner. While directly learning on real robots is appealing, it comes along with substantial challenges, such as high sample cost, partial observability, and safety constraints [28]. Hence, simulators are often

show abstract

Section: Learning From Demonstrationmentioning

confidence: 99%

Section: Key Insights On Guided Rl Compliancementioning

confidence: 99%

Guided Reinforcement Learning: A Review and Evaluation for Efficient and Effective Real-World Robotics [Survey]

Eßer

Bach

Jestel

et al. 2023

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

show abstract

“…The traditional BC algorithms typically learn an explicit model that maps from the state space to the action space in the framework of supervised learning. Such an approach has been applied to Autonomous driving [9] and robotic manipulation [10], [11], [12], [13], [14]. Recently, implicit behavior cloning [3] is the first behavioral cloning algorithm to use an energybased model (EBM), showing impressive results for learning discontinuous behaviors in robotic manipulation.…”

Section: Related Workmentioning

confidence: 99%

SEIL: Simulation-augmented Equivariant Imitation Learning

Jia¹,

Wang²,

Guanang³

et al. 2022

Preprint

View full text Add to dashboard Cite

In robotic manipulation, acquiring samples is extremely expensive because it often requires interacting with the real world. Traditional image-level data augmentation has shown the potential to improve sample efficiency in various machine learning tasks. However, image-level data augmentation is insufficient for an imitation learning agent to learn good manipulation policies in a reasonable amount of demonstrations. We propose Simulation-augmented Equivariant Imitation Learning (SEIL), a method that combines a novel data augmentation strategy of supplementing expert trajectories with simulated transitions and an equivariant model that exploits the O(2) symmetry in robotic manipulation. Experimental evaluations demonstrate that our method can learn non-trivial manipulation tasks within ten demonstrations and outperforms the baselines with a significant margin.

show abstract

“…Breyer et al ( 46) infer grasps at a resolution of just 40 3 . James et al circumvent this problem using hierarchy (48) in the context of Q learning. Specifically, they represent the Q function as a hierarchy of two Q models which we denote as Qcoarse and Q f ine .…”

Section: James Et Al (2022)mentioning

confidence: 99%

Grasp Learning: Models, Methods, and Performance

Platt¹

2022

Preprint

View full text Add to dashboard Cite

Grasp learning has become an exciting and important topic in robotics. Just a few years ago, the problem of grasping novel objects from unstructured piles of clutter was considered a serious research challenge. Now, it is a capability that is quickly becoming incorporated into industrial supply chain automation. How did that happen? What is the current state of the art in robotic grasp learning, what are the different methodological approaches, and what machine learning models are used? This review attempts to give an overview of the current state of the art of grasp learning research.

show abstract

Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisation

Cited by 35 publications

References 18 publications

Guided Reinforcement Learning: A Review and Evaluation for Efficient and Effective Real-World Robotics [Survey]

Guided Reinforcement Learning: A Review and Evaluation for Efficient and Effective Real-World Robotics [Survey]

SEIL: Simulation-augmented Equivariant Imitation Learning

Grasp Learning: Models, Methods, and Performance

Contact Info

Product

Resources

About