GE-Grasp: Efficient Target-Oriented Grasping in Dense Clutter

Li, Zhan; Wang, Ziwei; Huang, Sichao; Zhou, Jie; Lu, Jiwen

doi:10.1109/iros47612.2022.9981499

Cited by 11 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Liu et al. [38] proposed GE‐Grasp for grasping target objects using a generator‐evaluator architecture. These methods use push operations to reduce the clutter of the environment, thereby decreasing the interference of the complex environment with the grasping affordance maps.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, deep reinforcement learning based push and grasp synergy has emerged as a promising technique in cluttered environments [11,14,16,[35][36][37][38]. The authors of ref.…”

Section: Pushing and Graspingmentioning

confidence: 99%

“…Yang et al [37] proposed a non-maximum suppression policy (pol-icyNMS) to suppress unreasonable actions and designed a PR-Net to predict the global reward for pushing actions. Liu et al [38] proposed GE-Grasp for grasping target objects using a generator-evaluator architecture. These methods use push operations to reduce the clutter of the environment, thereby decreasing the interference of the complex environment with the grasping affordance maps.…”

Section: Pushing and Graspingmentioning

confidence: 99%

See 2 more Smart Citations

Position‐aware pushing and grasping synergy with deep reinforcement learning in clutter

Zhao

Zuo

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

The positional information of objects is crucial to enable robots to perform grasping and pushing manipulations in clutter. To effectively perform grasping and pushing manipulations, robots need to perceive the position information of objects, including the coordinates and spatial relationship between objects (e.g., proximity, adjacency). The authors propose an end‐to‐end position‐aware deep Q‐learning framework to achieve efficient collaborative pushing and grasping in clutter. Specifically, a pair of conjugate pushing and grasping attention modules are proposed to capture the position information of objects and generate high‐quality affordance maps of operating positions with features of pushing and grasping operations. In addition, the authors propose an object isolation metric and clutter metric based on instance segmentation to measure the spatial relationships between objects in cluttered environments. To further enhance the perception capacity of position information of the objects, the authors associate the change in the object isolation metric and clutter metric in cluttered environment before and after performing the action with reward function. A series of experiments are carried out in simulation and real‐world which indicate that the method improves sample efficiency, task completion rate, grasping success rate and action efficiency compared to state‐of‐the‐art end‐to‐end methods. Noted that the authors’ system can be robustly applied to real‐world use and extended to novel objects. Supplementary material is available at https://youtu.be/NhG\_k5v3NnM}{https://youtu.be/NhG\_k5v3NnM.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Recently, deep reinforcement learning based push and grasp synergy has emerged as a promising technique in cluttered environments [11,14,16,[35][36][37][38]. The authors of ref.…”

Section: Pushing and Graspingmentioning

confidence: 99%

Section: Pushing and Graspingmentioning

confidence: 99%

See 1 more Smart Citation

Position‐aware pushing and grasping synergy with deep reinforcement learning in clutter

Zhao

Zuo

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

show abstract

“…Robotic packing systems [21], [1], [34], [41], [13], [14] play a key role in warehouse automation with the benefits of reduced uptime, high throughput, and low accident rate compared with the labor-intensive approaches. The goal of robotic packing is to stow objects into constrained space such as shipping boxes.…”

Section: Introductionmentioning

confidence: 99%

Category-level Shape Estimation for Densely Cluttered Objects

Wu¹,

Wang²,

Lu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Accurately estimating the shape of objects in dense clutters makes important contribution to robotic packing, because the optimal object arrangement requires the robot planner to acquire shape information of all existed objects. However, the objects for packing are usually piled in dense clutters with severe occlusion, and the object shape varies significantly across different instances for the same category. They respectively cause large object segmentation errors and inaccurate shape recovery on unseen instances, which both degrade the performance of shape estimation during deployment. In this paper, we propose a category-level shape estimation method for densely cluttered objects. Our framework partitions each object in the clutter via the multi-view visual information fusion to achieve high segmentation accuracy, and the instance shape is recovered by deforming the category templates with diverse geometric transformations to obtain strengthened generalization ability. Specifically, we first collect the multi-view RGB-D images of the object clutters for point cloud reconstruction. Then we fuse the feature maps representing the visual information of multi-view RGB images and the pixel affinity learned from the clutter point cloud, where the acquired instance segmentation masks of multi-view RGB images are projected to partition the clutter point cloud. Finally, the instance geometry information is obtained from the partially observed instance point cloud and the corresponding category template, and the deformation parameters regarding the template are predicted for shape estimation. Experiments in the simulated environment and real world show that our method achieves high shape estimation accuracy for densely cluttered everyday objects with various shapes.

show abstract