Image retrieval using Markov Random Fields and global image features

Abstract

In this paper, we propose a direct image retrieval framework based on Markov Random Fields (MRFs) that exploits the semantic context dependencies of the image. The novelty of our approach lies in the use of different kernels in our non-parametric density estimation together with the utilisation of configurations that explore semantic relationships among concepts at the same time as low-level features, instead of just focusing on correlation between image features like in previous formulations. Hence, we introduce several configurations and study which one achieve the best performance. Results are presented for two datasets, the usual benchmark Corel 5k and the collection proposed by the 2009 edition of the ImageCLEF campaign. We observe that, using MRFs, performance increases significantly depending on the kernel used in the density estimation for the two datasets. With respect to the the language model, best results are obtained for the configuration that exploits dependencies between words together with dependencies between words and visual features. For the Corel 5k dataset, our best result corresponds to a mean average precision of 0.32, which compares favourably with the highest value ever obtained, 0.35, achieved by Makadia et al. [22] albeit with different features. For the ImageCLEF09 collection, we obtained 0.32, as mean average precision.

A Hybrid Model for Automatic Image Annotation

“…Nakayama et al [12] focused on a distance called canonical contextual distance (CCD) and applied it to image annotation task. Feng et al [4] and Llorente et al [10] formulated the image annotation problem as a multi label ranking problem. Wu et al [17] addressed the problem of class imbalance and weak labeling problem by a tag-completion technique for the training dataset based on some optimization criteria.…”

An energy-based model for region-labeling

“…Finally, we would like to mention that nowadays there is a research trend on the development of models that can take advantage of context through adaptive strategies for diverse computer vision tasks (i.e., models that are similar to ours) [20][21][22][23][24]. For example, Llorente et al developed a Markov random field model for image retrieval using word association information [20]; Jiang et al proposed a graph diffusion formulation that incorporates contextual information for video annotation [21]; Lee and Grauman proposed a graph based approach to object recognition that incorporates appearance and object-object relationships to discover new object categories [22]; Llorente et al explored the use of contextual models based on co-occurrence statistics for image annotation at the image-level [23]; Yao et al developed a random field model that incorporates human poses and object association information for recognition of human-object interaction activities (sports) [24].…”

“…For example, Llorente et al developed a Markov random field model for image retrieval using word association information [20]; Jiang et al proposed a graph diffusion formulation that incorporates contextual information for video annotation [21]; Lee and Grauman proposed a graph based approach to object recognition that incorporates appearance and object-object relationships to discover new object categories [22]; Llorente et al explored the use of contextual models based on co-occurrence statistics for image annotation at the image-level [23]; Yao et al developed a random field model that incorporates human poses and object association information for recognition of human-object interaction activities (sports) [24]. This sample of successful contextual methods are evidence that the use of context is beneficial for computer vision and image understanding.…”

Review of image low-level feature extraction methods for content-based image retrieval

“…statistical, model, structure and signal. Commonly used texture description algorithms are the gray level co-occurrence matrix (GLCM) (Haralick, 1973; Mukherjee, 2016; Zhang et al , 2008), the Markov random field (MRF) (Cross and Jain, 1983; André Ricardo et al , 2015; Llorente et al , 2010), the wavelet transform (WT) (Mallat, 1989; Ashraf et al , 2018; Sadafale and Bonde, 2017), and the Gabor filters (Ma and Manjunath, 1996; Barbu, 2009; Chawki et al , 2017). The GLCM is one of the most famous texture analysis methods.…”