The Topology and Language of Relationships in the Visual Genome Dataset

Chacra, David Abou; Zelek, John

doi:10.1109/cvprw56347.2022.00533

Cited by 5 publications

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conversely, triplets such as <man, standing on, street> and <street, under, man> are considered as incorrect predictions and are treated as negative examples during training, despite being semantically valid triplets. The presence of rare, bidirectional [65], and unannotated relationships [46] hinders the learning of representative semantics for SGG methods. Similar to the RCA discussed in Section 3.3.4, Q L and K L are projected with h A heads, and each head has d A dimensions.…”

Section: Relationships As Attention Weightsmentioning

confidence: 99%

Fully Convolutional Scene Graph Generation

Liu

Yan

Mortazavi

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Scene Graph Generation (SGG) has achieved significant progress recently. However, most previous works rely heavily on fixed-size entity representations based on bounding box proposals, anchors, or learnable queries. As each representation's cardinality has different trade-offs between performance and computation overhead, extracting highly representative features efficiently and dynamically is both challenging and crucial for SGG. In this work, a novel architecture called RepSGG is proposed to address the aforementioned challenges, formulating a subject as queries, an object as keys, and their relationship as the maximum attention weight between pairwise queries and keys. With more fine-grained and flexible representation power for entities and relationships, RepSGG learns to sample semantically discriminative and representative points for relationship inference. Moreover, the long-tailed distribution also poses a significant challenge for generalization of SGG. A run-time performance-guided logit adjustment (PGLA) strategy is proposed such that the relationship logits are modified via affine transformations based on run-time performance during training. This strategy encourages a more balanced performance between dominant and rare classes. Experimental results show that RepSGG achieves the state-of-the-art or comparable performance on the Visual Genome and Open Images V6 datasets with fast inference speed, demonstrating the efficacy and efficiency of the proposed methods.

show abstract

Section: Relationships As Attention Weightsmentioning

confidence: 99%