Visual Question Answering over Scene Graph

Visual Question Answering based on Formal Logic

“…When the validation set is limited to those questions which have a consistent functional form, then the validation accuracy becomes (93%), which is a better indicator of the model's performance on the GQA datasets. This is comparable to the current state of the art for the GQA dataset under perfect site configuration [9] (96.3%). Aside from the fact that the proposed solution is also highly interpretable and the reasoning steps are easy to follow, the training of the transformer is much more easier to perform than a Graph Neural Network used in [9].…”

“…This is comparable to the current state of the art for the GQA dataset under perfect site configuration [9] (96.3%). Aside from the fact that the proposed solution is also highly interpretable and the reasoning steps are easy to follow, the training of the transformer is much more easier to perform than a Graph Neural Network used in [9]. Table IV summarizes the model performance on the GQA dataset.…”

“…Since a scene graph is a symbolic representation of a scene, this opens up the possibility of performing symbolic reasoning on images. Building on this, [8] show how scene graphs can be utilized for image captioning and [9] demonstrate the use of scene graphs in their VQA pipeline. In particular, [9] make use of Graph Neural Networks on top of the scene graph to create higher level abstractions, making it not very easily trainable.…”

“…Building on this, [8] show how scene graphs can be utilized for image captioning and [9] demonstrate the use of scene graphs in their VQA pipeline. In particular, [9] make use of Graph Neural Networks on top of the scene graph to create higher level abstractions, making it not very easily trainable. In this paper we directly operate on the scene graph How many blocks are red metal things or big green metal things?…”

PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation

“…Scene graph generation (SGG) [13], which is a visual task to detect objects and recognize semantic relationships between different objects in an image, can serve as a powerful structural representation of images and benefit other high-level Vision-and-Language tasks such as image generation [12,33,40], image retrieval [13,24,28,34], visual question answering [6,18,39] and image captioning [7,19,38]. Taking advantage of the remarkable feature representations of convolutional neural networks (CNNs) [16] and diverse contextual feature fusion strategies (e.g., message passing [20,37], lstm [41]), a variety of methods have made significant progress to improve the recall evaluation metric performance of SGG tasks.…”