Configurable Graph Reasoning for Visual Relationship Detection

Optimizing Continuous Prompts for Visual Relationship Detection by Affix-Tuning

“…In recent years, Zhu et al [12] proposed configurable graph reasoning (CGR) to decompose the reasoning path of visual relationships and the incorporation of external knowledge, achieving configurable knowledge selection and personalizing graph reasoning for each relationship type in each image. In addition, given a common sense knowledge graph, it adaptively configures the reasoning path based on the knowledge graph, bridges the semantic gap between the common sense knowledge and real world scenes, and achieves better knowledge generalization [12]. Hung et al [9] proposed a context-augmented translation embedding model that could capture both common and rare relationships.…”

“…VRD contains 5000 images (4000 for training and 1000 for testing) with 100 object categories and 70 predicates. In total, it contains 37,993 relationships with 6,672 relations and 24.25 predicates per object category [12]. Visual Genome (VG) contains a large number of images with content semantic information that is richer than ImageNet, released by Stanford University in 2015.…”

Visual Navigation Based on Language Assistance and Memory

“…Visual language navigation can get rid of the dependence of map navigation in the process of mobile robots moving and can guide the movement of mobile robots according to the method of integrating language description and scenes. Visual relationship detection (VRD) [3] is applied to extract the scene features of objects to determine landmarks and select how to execute action. At present, visual language navigation system is based on single language description, which is suitable for short distance navigation.…”