Semantic Scene Classification for Image Annotation and Retrieval

Abstract

Abstract. We describe an annotation and retrieval framework that uses a semantic image representation by contextual modeling of images using occurrence probabilities of concepts and objects. First, images are segmented into regions using clustering of color features and line structures. Next, each image is modeled using the histogram of the types of its regions, and Bayesian classifiers are used to obtain the occurrence probabilities of concepts and objects using these histograms. Given the observation that a single class with the highest probability is not sufficient to model image content in an unconstrained data set with a large number of semantically overlapping classes, we use the concept/object probabilities as a new representation, and perform retrieval in the semantic space for further improvement of the categorization accuracy. Experiments on the TRECVID and Corel data sets show good performance.

Automatic Image Annotation Using Color K-Means Clustering

“…With each successive pass, a data value can change the centroid where it belongs to, thus altering the values of the centroid at every pass. KMeans clustering has been used extensively to facilitate in classifications of low-level features in image retrieval systems [3][14] [11][7] [9]. The EM algorithm employed in [13] [14], on the other hand relies on soft assignment of data given set of centroids.…”