GPS-Net: Graph Property Sensing Network for Scene Graph Generation

Abstract

lem, we mitigate this issue by first softening the distribution and then enabling it to be adjusted for each subjectobject pair according to their visual appearance. Systematic experiments demonstrate the effectiveness of the proposed techniques. Moreover, GPS-Net achieves state-ofthe-art performance on three popular databases: VG, OI, and VRD by significant gains under various settings and metrics. The code and models are available at https: //github.com/taksau/GPS-Net.

BGT-Net: Bidirectional GRU Transformer Network for Scene Graph Generation

“…This bias arrives from the long-tailed relationship distribution. The GPS-Net [21] tackled this problem with FS and BA which worked well compared to the previous works. The overall performance of the model could be improved as well as improvements in mean Recall@K were achieved, which gives reasoning about the positive effect of their approach in handling the dataset bias.…”

“…In the less common two-stage approach [33,10,4,33], attributes of the scene graph are used in the second training step to refine the results produced by the first stage. Much more common are the one-stage approaches [4,45,5,37,39,21,18,22,17,24] which focus only on object detection and relationship classification, while almost neglecting intrinsic features. The proposed BGT-Net follows a one step approach and has the following advantages as compared to the literature work: (1) It uses object-object communication which improves the performance in SGG;…”

“…To handle the long-tailed relationship distribution present in the Visual Genome dataset, the procedure of softening this distribution and adapting the bias term for every subjectobject pair form [21] is adopted. The used features in this step is different than in the GPS-Net [21] , but the principle stays the same. The softening of the relationship distribution is done by applying a log-softmax function to the original distribution.…”

“…In this procedure, a log-softmax function is applied to the subjectobject pairwise relationship distribution. Following this Frequency Softening (FS), a Bias Adaptation (BA) approach [21] is used. The bias for every subject-object is controlled by the bias adaptation term which takes scenespecific inputs to vary the amount of added bias.…”

“…(3) For every object, a second transformer encoder is used to gather information for the edges. (4) To tackle the bias in the relationship distribution, FS and BA [21] is adopted. The evaluation efficacy of the proposed BGT-Net is performed on three SGG datasets: Visual Genome (VG) [13], OpenImages (OI) [14], and Visual Relationship Detection (VRD) [22].…”