Lightweight Pixel-Wise Generative Robot Grasping Detection Based on RGB-D Dense Fusion

Tian, Hongkun; Song, Kechen; Li, Song; Ma, Shuai; Yan, Yunhui

doi:10.1109/tim.2022.3196130

Cited by 17 publications

(12 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in the figure, the quality heatmaps demonstrate the robustness of our proposed method, which contributes to the superior performance of our grasp detection results. We also conducted a comparative analysis of our grasp detection algorithm with that of several other methods [5,6,[8][9][10][11][12] using the Jacquard dataset. Table 3 presents the statistical results of our experiment with the Jacquard dataset.…”

Section: Cornell Dataset Experiments Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Zhang¹,

Sun²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

In the field of vision-based robot grasping, effectively leveraging RGB and depth information to accurately determine the position and pose of a target is a critical issue. To address this challenge, we propose a tri-stream cross-modal fusion architecture for 2-DoF visual grasp detection. This architecture facilitates the interaction of RGB and depth bilateral information and is designed to efficiently aggregate multiscale information. Our novel modal interaction module (MIM) with spatial-wise cross-attention algorithm adaptively captures cross-modal feature information. Meanwhile, the channel interaction modules (CIM) further enhance the aggregation of different modal streams. In addition, we efficiently aggregate global multiscale information through a hierarchical structure with skipping connections. To evaluate the performance of our proposed method, we conduct validation experiments on standard public datasets and real robot grasping experiments. We achieve the image-wise detection accuracy of 99.4% and 96.7% on Cornell and Jacquard datasets respectively. The object-wise detection accuracy reaches 97.8% and 94.6% on the same datasets. Furthermore, physical experiments using the 6-DoF Elite robot demonstrate a success rate of 94.5%. These experiments highlight the superior accuracy of our proposed method.

show abstract

Section: Cornell Dataset Experiments Resultsmentioning

confidence: 99%

“…H. Tian et al [11] introduced an intermediate-fusion method for lightweight pixelwise robot grasp detection, utilizing RGB and depth information. In 2023, H. Tian et al [12] extended their work by introducing a rotation adaptive grasp detection approach, which also utilizes intermediate data fusion.…”

Section: Multiple Modality Fusion Based Grasp Detectionmentioning

confidence: 99%

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Zhang¹,

Sun²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In their work, a hierarchical weighted suppress interference approach was introduced to achieve robust features. H. Tian et al [ 11 ] introduced an intermediate-fusion method for lightweight pixel-wise robot grasp detection, utilizing RGB and depth information. In 2023, H. Tian et al [ 12 ] extended their work by introducing a rotation adaptive grasp detection approach, which also utilizes intermediate data fusion.…”

Section: Related Workmentioning

confidence: 99%

“…It is our hypothesis that the feature aggregation process should optimally utilize useful information from all three features while minimizing the impact of irrelevant information. Previous research, such as that conducted in [ 2 , 11 ], has made numerous attempts to explore this topic. However, both studies employ equal-weight feature aggregation for each channel.…”

Section: Approachmentioning

confidence: 99%

“…While these methods have yielded meaningful results, the correlation between multimodal data has only been partially exploited. Recent studies have focused on exploring the mechanisms of intermediate fusion [ 11 , 12 ]. Although these methods have improved the efficiency of RGB and depth modalities in robot grasp detection, making the most of the bilateral modal information still remains a challenge.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Zhang

XueYing

2023

Sensors

View full text Add to dashboard Cite

In the field of vision-based robot grasping, effectively leveraging RGB and depth information to accurately determine the position and pose of a target is a critical issue. To address this challenge, we proposed a tri-stream cross-modal fusion architecture for 2-DoF visual grasp detection. This architecture facilitates the interaction of RGB and depth bilateral information and was designed to efficiently aggregate multiscale information. Our novel modal interaction module (MIM) with a spatial-wise cross-attention algorithm adaptively captures cross-modal feature information. Meanwhile, the channel interaction modules (CIM) further enhance the aggregation of different modal streams. In addition, we efficiently aggregated global multiscale information through a hierarchical structure with skipping connections. To evaluate the performance of our proposed method, we conducted validation experiments on standard public datasets and real robot grasping experiments. We achieved image-wise detection accuracy of 99.4% and 96.7% on Cornell and Jacquard datasets, respectively. The object-wise detection accuracy reached 97.8% and 94.6% on the same datasets. Furthermore, physical experiments using the 6-DoF Elite robot demonstrated a success rate of 94.5%. These experiments highlight the superior accuracy of our proposed method.

show abstract

Antipodal-points-aware dual-decoding network for robotic visual grasp detection oriented to multi-object clutter scenes

Tian

Song

et al. 2023

Expert Systems with Applications

View full text Add to dashboard Cite

Lightweight Pixel-Wise Generative Robot Grasping Detection Based on RGB-D Dense Fusion

Cited by 17 publications

References 33 publications

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Antipodal-points-aware dual-decoding network for robotic visual grasp detection oriented to multi-object clutter scenes

Contact Info

Product

Resources

About