RSSGG_CS: Remote Sensing Image Scene Graph Generation by Fusing Contextual Information and Statistical Knowledge

Abstract

To semantically understand remote sensing images, it is not only necessary to detect the objects in them but also to recognize the semantic relationships between the instances. Scene graph generation aims to represent the image as a semantic structural graph, where objects and relationships between them are described as nodes and edges, respectively. Some existing methods rely only on visual features to sequentially predict the relationships between objects, ignoring contextual information and making it difficult to generate high-quality scene graphs, especially for remote sensing images. Therefore, we propose a novel model for remote sensing image scene graph generation by fusing contextual information and statistical knowledge, namely RSSGG_CS. To integrate contextual information and calculate attention among all objects, the RSSGG_CS model adopts a filter module (FiM) that is based on adjusted transformer architecture. Moreover, to reduce the blindness of the model when searching semantic space, statistical knowledge of relational predicates between objects from the training dataset and the cleaned Wikipedia text is used as supervision when training the model. Experiments show that fusing contextual information and statistical knowledge allows the model to generate more complete scene graphs of remote sensing images and facilitates the semantic understanding of remote sensing images.

Dynamic Scene Graph Generation of Point Clouds with Structural Representation Learning

“…A graph is used to describe the semantic label of objects and their relationships in the image, and it helps applications such as image retrieval. Later, there are lines of works promoting the study of scene graph generations in 2D computer vision [4,8,9] . Different approaches are proposed to generate scene graphs according to the understanding of images, such as a variant of Graph Convolutional Network (GCN) [10] and Long Short-Term Memory (LSTM)-based MotifNet [11] .…”

A State-of-the-Art Computer Vision Adopting Non-Euclidean Deep-Learning Models

“…Machining feature [105]: a synthetic labeled, balanced dataset representing machining features such as chamfers and circular end pockets applied to a cube. FabWave [106]: a small labeled, imbalanced collection of Salient object detection [110] Agile formation control of drone focking [111] Online semantic mapping system [112] Nonlinear estimation in robotics and vision [113] Degree of disease in citrus fruits [114] Network model suitable for indoor mobile robots [115] 3D object detection using point clouds [116] Detecting visual relations and scene parsing [117] Detecting 3D objects from noisy point clouds [118] Robust label propagation for saliency detection [119] Weakly supervised object detection [120] Detection of distinct objects like seed selection [121] Segmentation of rivers for autonomous surface vehicles [122] 3D scene perception [123] City object detection from airborne LiDAR data [124] Nonmaximal suppression product detection on the shelf [125] LiDAR-based three-dimensional mapping in urban environments [126] 3D multiobject tracking in point clouds 2D image processing [127] Topic scene graphs for image captioning [128] Text-based visual question answering [129] Single-image 3D reconstruction [130] Laser-based surface damage detection and quantifcation [131] Smooth manifold triangulation [132] Deep virtual stereo odometry Hyperspectral imaginary [133] Enhanced discriminative broad learning system [134] Spectral-spatial clustering of hyperspectral image [135] Geometric knowledge embedding [136] Hyperspectral and multispectral image fusion [137] Hyperspectral unmixing [138] Remote sensing image scene graph generation Medical imaging [139] Brain graph synth...…”