Graph-based Visual Manipulation Relationship Reasoning in Object-Stacking Scenes

Zuo, Guoyu; Tong, Jiayuan; Liu, Hongxing; Chen, Wenbai; Li, Jianfeng

doi:10.1109/ijcnn52387.2021.9534389

Cited by 2 publications

(5 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-task CNN by Zhang et al [18], followed by VMRN by Zhang et al [19], are CNN-based approaches that were trained on VMRD. Zuo et al [3] proposed GVMRN, a hybrid network of CNN and GNN (Graph Neural Network). Future papers may use the REGRAD simulated image dataset [20] to train models that infer object hierarchy effectively.…”

Section: Related Workmentioning

confidence: 99%

“…We compared the performance of our algorithm on the VMRD dataset against current state-of-the-art approaches for object-stacking detection: Multi-task CNN [18], VMRN [19] and GVMRN [3]. We considered a relationship inference to be correct if:…”

Section: B Adj-net Performancementioning

confidence: 99%

“…The maximum information entropy parameter considered for the experiment is H th = 0.45. We perform bin picking in a setting with various types of objects in various Models OR OP IA Multi-task CNN [18] 86.0 88.8 67.1 VMRN-RN101 [19] 85.4 85.5 65.8 VMRN-VGG16 [19] 86.3 88.8 68.4 GVMRN-RF-RN101 [3] 86.9 87.5 68.8 GVMRN-RF-VGG16 [3] 88.…”

Section: Robot Experiments Setupmentioning

confidence: 99%

“…IA-2 IA-3 IA-4 IA-5 Multi-task CNN [18] 87.7 64.1 56.6 72.9 GVMRN-RF-RN101 [3] 91.4 69.2 61.2 57.5 GVMRN-RF-VGG16 [3] 92.9 70. setups: office supplies, kitchen tools, various paste tubes and liquid bottles, and package boxes, all presented in Fig. 5.…”

Section: Modelsmentioning

confidence: 99%

“…On the one hand, current stateof-the-art deep-learning-based approaches (e.g. [3]) assume the hierarchy based on a single image and do not take into account the uncertainty in object hierarchy. These approaches are, therefore, error prone and do not inherently support multi-view inference.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

DUQIM-Net: Probabilistic Object Hierarchy Representation for Multi-View Manipulation

Turetsky¹,

Miron²,

Di-Castro³

2022

Preprint

View full text Add to dashboard Cite

Object manipulation in cluttered scenes is a difficult and important problem in robotics. To efficiently manipulate objects, it is crucial to understand their surroundings, especially in cases where multiple objects are stacked one on top of the other, preventing effective grasping. We here present DUQIM-Net, a decision-making approach for object manipulation in a setting of stacked objects. In DUQIM-Net, the hierarchical stacking relationship is assessed using Adj-Net, a model that leverages existing Transformer Encoder-Decoder object detectors by adding an adjacency head. The output of this head probabilistically infers the underlying hierarchical structure of the objects in the scene. We utilize the properties of the adjacency matrix in DUQIM-Net to perform decision making and assist with object-grasping tasks. Our experimental results show that Adj-Net surpasses the state-of-the-art in object-relationship inference on the Visual Manipulation Relationship Dataset (VMRD), and that DUQIM-Net outperforms comparable approaches in bin clearing tasks.

show abstract