Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation

Li, Rongjie; Zhang, Songyang; Wan, Bo; He, Xuming

doi:10.1109/cvpr46437.2021.01096

Cited by 139 publications

(129 citation statements)

References 37 publications

Supporting

Mentioning

129

Contrasting

Order By: Relevance

“…Although R@50 of RelTR is 10.87 points lower than the best two-stage method VCTree [37], RelTR has the highest wmAP 𝑟 𝑒𝑙 (0.66 points higher than BGNN [46]) and wmAP 𝑝ℎ𝑟 (3.13 points higher than VCTree [37]). The final weighted score of RelTR is only 0.43 points lower than the best two-stage model, which is a very small performance gap.…”

Section: Open Images V6mentioning

confidence: 88%

“…Our model is also competitive compared with recent two-stage models, and outperforms state-of-the-art visual-based methods. Although the R@20/R@50 score of RelTR is 3.1/5.8 points lower than that of BGNN [46], RelTR is a light-weight model, which has only 63.7M parameters and an inference speed of 16.6 FPS, ca. 7 times faster than BGNN.…”

Section: Visual Genomementioning

confidence: 90%

“…Xu et al [42] use standard RNNs to iteratively improves the relationship prediction via message passing while MotifNet [9] stacks LSTMs to reason about the local and global context. Graphbased models [44], [45], [46], [47], [48] perform message passing and demonstrate good results. Factorizable Net [45] decomposes and combines the graphs to infer the relationships.…”

Section: Scene Graph Generationmentioning

confidence: 97%

“…To further demonstrate the performance of RelTR, we compare the average precision (AP) of relationships and phrases for RelTR and BGNN [46] (see Fig. 5) with Open Images V6.…”

Section: Open Images V6mentioning

confidence: 99%

See 3 more Smart Citations

RelTR: Relation Transformer for Scene Graph Generation

Yang¹,

Yang²,

Rosenhahn³

2022

Preprint

View full text Add to dashboard Cite

Different objects in the same scene are more or less related to each other, but only a limited number of these relationships are noteworthy. Inspired by DETR, which excels in object detection, we view scene graph generation as a set prediction problem and propose an end-to-end scene graph generation model RelTR which has an encoder-decoder architecture. The encoder reasons about the visual feature context while the decoder infers a fixed-size set of triplets subject-predicate-object using different types of attention mechanisms with coupled subject and object queries. We design a set prediction loss performing the matching between the ground truth and predicted triplets for the end-to-end training. In contrast to most existing scene graph generation methods, RelTR is a one-stage method that predicts a set of relationships directly only using visual appearance without combining entities and labeling all possible predicates. Extensive experiments on the Visual Genome and Open Images V6 datasets demonstrate the superior performance and fast inference of our model.

show abstract

Section: Open Images V6mentioning

confidence: 88%

Section: Visual Genomementioning

confidence: 90%

Section: Scene Graph Generationmentioning

confidence: 97%

“…To further demonstrate the performance of RelTR, we compare the average precision (AP) of relationships and phrases for RelTR and BGNN [46] (see Fig. 5) with Open Images V6.…”

Section: Open Images V6mentioning

confidence: 99%

See 2 more Smart Citations

RelTR: Relation Transformer for Scene Graph Generation

Yang¹,

Yang²,

Rosenhahn³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Tang et al also try to use causal analysis (Tang et al 2020) to reduce the influence of training data distribution on the final model. Some other works (Chen et al 2019a;Zhan et al 2020;Chiou et al 2021) address this issue in a positive-unlabeled learning manner, and typical imbalance learning methods, such as re-sampling and costsensitive learning, are also introduced for scene graph generation (Li et al 2021a;Yan et al 2020). Unlike these approaches, we adopt the resistance training strategy using prior bias (RTPB), which utilizes a resistance bias item for the relationship classifier during training to optimize the loss value and the classification margin of each type of relationship.…”

Section: Scene Graph Generationmentioning

confidence: 99%

Resistance Training using Prior Bias: toward Unbiased Scene Graph Generation

Chen¹,

Zhan²,

Yu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Scene Graph Generation (SGG) aims to build a structured representation of a scene using objects and pairwise relationships, which benefits downstream tasks. However, current SGG methods usually suffer from sub-optimal scene graph generation because of the long-tailed distribution of training data. To address this problem, we propose Resistance Training using Prior Bias (RTPB) for the scene graph generation. Specifically, RTPB uses a distributed-based prior bias to improve models' detecting ability on less frequent relationships during training, thus improving the model generalizability on tail categories. In addition, to further explore the contextual information of objects and relationships, we design a contextual encoding backbone network, termed as Dual Transformer (DTrans). We perform extensive experiments on a very popular benchmark, VG150, to demonstrate the effectiveness of our method for the unbiased scene graph generation. In specific, our RTPB achieves an improvement of over 10% under the mean recall when applied to current SGG methods. Furthermore, DTrans with RTPB outperforms nearly all state-ofthe-art methods with a large margin. 1 * This work was done when Chao Chen was a research intern at JD Explore Academy.† Corrsponding Author.

show abstract

Hierarchical Memory Learning for Fine-Grained Scene Graph Generation

Deng

Zhang

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation

Cited by 139 publications

References 37 publications

RelTR: Relation Transformer for Scene Graph Generation

RelTR: Relation Transformer for Scene Graph Generation

Resistance Training using Prior Bias: toward Unbiased Scene Graph Generation

Hierarchical Memory Learning for Fine-Grained Scene Graph Generation

Contact Info

Product

Resources

About