Real-Time Perception Meets Reactive Motion Generation

Kappler, Daniel; Meier, Franziska; Issac, Jan; Mainprice, Jim; Cifuentes, Cristina Garcia; Wüthrich, Manuel; Berenz, Vincent; Schaal, Stefan; Ratliff, Nathan; Bohg, Jeannette

doi:10.1109/lra.2018.2795645

Cited by 83 publications

(63 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Closed-loop control of a robot to a desired pose using visual feedback is commonly referred to as visual servoing (Hutchinson et al, 1996). The advantages of visual servoing and other closed-loop methods are that they are able to adapt to dynamic environments and do not necessarily require fully accurate camera calibration or position control (Kappler et al, 2018). A number of works apply visual servoing directly to grasping applications, with a survey given by Kragic and Christensen (2002).…”

Section: Related Workmentioning

confidence: 99%

Learning robust, real-time, reactive robotic grasping

Morrison

Corke

Leitner

2019

The International Journal of Robotics Research

304

236

View full text Add to dashboard Cite

We present a novel approach to perform object-independent grasp synthesis from depth images via deep neural networks. Our generative grasping convolutional neural network (GG-CNN) predicts a pixel-wise grasp quality that can be deployed in closed-loop grasping scenarios. GG-CNN overcomes shortcomings in existing techniques, namely discrete sampling of grasp candidates and long computation times. The network is orders of magnitude smaller than other state-of-the-art approaches while achieving better performance, particularly in clutter. We run a suite of real-world tests, during which we achieve an 84% grasp success rate on a set of previously unseen objects with adversarial geometry and 94% on household items. The lightweight nature enables closed-loop control of up to 50 Hz, with which we observed 88% grasp success on a set of household objects that are moved during the grasp attempt. We further propose a method combining our GG-CNN with a multi-view approach, which improves overall grasp success rate in clutter by 10%. Code is provided at https://github.com/dougsm/ggcnn

show abstract

Section: Related Workmentioning

confidence: 99%

Learning robust, real-time, reactive robotic grasping

Morrison

Corke

Leitner

2019

The International Journal of Robotics Research

304

236

View full text Add to dashboard Cite

show abstract

“…As future work, our goal is to execute these policies on a real robot equipped with the same hands. This would be an extension of our dynamic manipulation system [11] that would allow it to perform a larger set of grasps on the visually tracked objects. This requires an adaptation of the assumed contact sensing to make it more similar to the hardware.…”

Section: Discussionmentioning

confidence: 99%

Leveraging Contact Forces for Learning to Grasp

Merzic

Bogdanovic

Kappler

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Grasping objects under uncertainty remains an open problem in robotics research. This uncertainty is often due to noisy or partial observations of the object pose or shape. To enable a robot to react appropriately to unforeseen effects, it is crucial that it continuously takes sensor feedback into account. While visual feedback is important for inferring a grasp pose and reaching for an object, contact feedback offers valuable information during manipulation and grasp acquisition. In this paper, we use model-free deep reinforcement learning to synthesize control policies that exploit contact sensing to generate robust grasping under uncertainty. We demonstrate our approach on a multi-fingered hand that exhibits more complex finger coordination than the commonly used twofingered grippers. We conduct extensive experiments in order to assess the performance of the learned policies, with and without contact sensing. While it is possible to learn grasping policies without contact sensing, our results suggest that contact feedback allows for a significant improvement of grasping robustness under object pose uncertainty and for objects with a complex shape.

show abstract

“…When the focus is more on efficient and simple manipulation, [80] shows that a tight coupling between realtime perception and reactive motion planning can lead to fast and accurate manipulation. Another, very cost effective approach has been proposed in [51].…”

Section: Avoiding Collisions With the Environmentmentioning

confidence: 99%

The State of Lifelong Learning in Service Robots:

Kasaei

Melsen

Beers

et al. 2021

J Intell Robot Syst

View full text Add to dashboard Cite

Service robots are appearing more and more in our daily life. The development of service robots combines multiple fields of research, from object perception to object manipulation. The state-of-the-art continues to improve to make a proper coupling between object perception and manipulation. This coupling is necessary for service robots not only to perform various tasks in a reasonable amount of time but also to continually adapt to new environments and safely interact with non-expert human users. Nowadays, robots are able to recognize various objects, and quickly plan a collision-free trajectory to grasp a target object in predefined settings. Besides, in most of the cases, there is a reliance on large amounts of training data. Therefore, the knowledge of such robots is fixed after the training phase, and any changes in the environment require complicated, time-consuming, and expensive robot re-programming by human experts. Therefore, these approaches are still too rigid for real-life applications in unstructured environments, where a significant portion of the environment is unknown and cannot be directly sensed or controlled. In such environments, no matter how extensive the training data used for batch learning, a robot will always face new objects. Therefore, apart from batch learning, the robot should be able to continually learn about new object categories and grasp affordances from very few training examples on-site. Moreover, apart from robot self-learning, non-expert users could interactively guide the process of experience acquisition by teaching new concepts, or by correcting insufficient or erroneous concepts. In this way, the robot will constantly learn how to help humans in everyday tasks by gaining more and more experiences without the need for re-programming. In this paper, we review a set of previously published works and discuss advances in service robots from object perception to complex object manipulation and shed light on the current challenges and bottlenecks.

show abstract

Real-Time Perception Meets Reactive Motion Generation

Cited by 83 publications

References 40 publications

Learning robust, real-time, reactive robotic grasping

Learning robust, real-time, reactive robotic grasping

Leveraging Contact Forces for Learning to Grasp

The State of Lifelong Learning in Service Robots:

Contact Info

Product

Resources

About