Image retrieval through qualitative representations over semantic features

Abstract

We propose a qualitative knowledge-driven semantic modelling approach for image retrieval based on qualitative relations over local semantic concepts of images. The relative similarity of two images is proportional to their qualitative similarity. The similarity measure is calculated for each query by exploiting the notion of conceptual neighbourhood -a measure of closeness between qualitative relations. The approach is motivated by the need to perform semantic querying using qualitative relations and bridge the semantic gap between a human user and that of CBIR systems. Three qualitative representations (and several variants) and a corpus of 700 natural scene images have been used to evaluate the effectiveness of image retrieval using this approach.

A Biologically Inspired Approach for Robot Depth Estimation

“…Oliva and Torralba [ 35 ] proposed a computational model for classifying perceptual properties (e.g., naturalness, openness, roughness, expansion, and ruggedness) into semantic categories such as street, highway, or coast. Going a step further, Qayyum and Cohn [ 36 ] used more than one concept to semantically describe landing images, while Lim and Chan [ 37 ] labelled images of natural scenes by means of a fuzzy qualitative approach. On their behalf, Buoncompagni et al [ 38 ] represented the robot environment using a fuzzy ontology.…”

Qualitative distances and qualitative description of images for indoor scene description and recognition in robotics

“…Related studies have been published that extract qualitative or semantic information from images representing scenes [Socher et al, 1997;Lovett et al, 2006;Qayyum and Cohn, 2007;Oliva and Torralba, 2001;Quattoni and Torralba, 2009]. Socher et al [1997] provide a verbal description of an image to a robotic manipulator system so it can identify and pick up an object that has been previously modelled geometrically and then categorized qualitatively by its type, colour, size and shape.…”

“…Lovett et al [2006] proposed a qualitative description for sketch image recognition, which describes lines, arcs and ellipses as basic elements and also the relative position, length and orientation of their edges. Qayyum and Cohn [2007] divide landscape images using a grid for their description so that semantic categories (grass, water, etc.) can be identified and qualitative relations of relative size, time and topology can be used for image description and retrieval in databases.…”

“…The approach of Lovett et al [2006] is applied to sketches, while our approach is applied to digital images captured from the real robot environment. Qayyum and Cohn [2007] use a grid to divide the image and describe what is inside each grid square (grass, water, etc. ), which is adequate for their application but the objects are divided in an artificial number of parts that depend on the size of the cell, while our approach extracts complete objects, which we think is more cognitive.…”