Bridging the Gap between Local Semantic Concepts and Bag of Visual Words for Natural Scene Image Retrieval

Abstract

This paper addresses the problem of semantic-based image retrieval of natural scenes. A typical content-based image retrieval system deals with the query image and images in the dataset as a collection of low-level features and retrieves a ranked list of images based on the similarities between features of the query image and features of images in the image dataset. However, top ranked images in the retrieved list, which have high similarities to the query image, may be different from the query image in terms of the semantic interpretation of the user which is known as the semantic gap. In order to reduce the semantic gap, this paper investigates how natural scene retrieval can be performed using the bag of visual word model and the distribution of local semantic concepts. The paper studies the efficiency of using different approaches for representing the semantic information, depicted in natural scene images, for image retrieval. An extensive experimental work has been conducted to study the efficiency of using semantic information as well as the bag of visual words model for natural and urban scene image retrieval.

Sir-Dl: An Architecture of Semantic-Based Image Retrieval Using Deep Learning Technique and RDF Triple Language

“…There were many techniques of multimedia retrieval by semantics that have been widely applied in many different fields such as query techniques on Ontology-based for the purpose of exact meaning interpretation of user query [19], visual encoding model based on convolutional neural network [31], semantic-based natural image retrieval using bag of visual word model and distribution of local semantic concepts [3], an efficient video retrieval based on semantic graph queries [12], an adaptive image search engine for deep knowledge and meaning of the image applied in Ontology-based to produce a new level of image meaning [18], content based semantics and image retrieval system for hierarchical databases [24], etc.…”

“…The general architecture of SIR-DL is described in Figure 1 and it is implemented by classifying images into visual word vectors based on deep learning network and performing image retrieval on RDF triple language. This model is built based on combination of components including deep learning network [16,20,21], BoVW technique [23,28,29], and semantic query on ontology in SPARQL language [3,8,18,26]. Based on deep learning, the classification model of semantic images is trained on dataset to create inputs for the problem of image retrieval on Ontology.…”