Unsupervised Semantic Feature Discovery for Image Object Retrieval and Tag Refinement

Abstract

Abstract-We have witnessed the exponential growth of images and videos with the prevalence of capture devices and the ease of social services such as Flickr and Facebook. Meanwhile, enormous media collections are along with rich contextual cues such as tags, geo-locations, descriptions, and time. To obtain desired images, users usually issue a query to a search engine using either an image or keywords. Therefore, the existing solutions for image retrieval rely on either the image contents (e.g., low-level features) or the surrounding texts (e.g., descriptions, tags) only. Those solutions usually suffer from low recall rates because small changes in lighting conditions, viewpoints, occlusions, or (missing) noisy tags can degrade the performance significantly. In this work, we tackle the problem by leveraging both the image contents and associated textual information in the social media to approximate the semantic representations for the two modalities. We propose a general framework to augment each image with relevant semantic (visual and textual) features by using graphs among images. The framework automatically discovers relevant semantic features by propagation and selection in textual and visual image graphs in an unsupervised manner. We investigate the effectiveness of the framework when using different optimization methods for maximizing efficiency. The proposed framework can be directly applied to various applications, such as keyword-based image search, image object retrieval, and tag refinement. Experimental results confirm that the proposed framework effectively improves the performance of these emerging image retrieval applications.

Instant messaging with emotion-embedded vectorized handwritings on mobile devices

“…Liu et al [3] survey the advances in online Chinese character recognition. Although there are more feature extraction algorithms [14,15] invented for image content understanding, developing proper features for Chinese characters is still problematic because of different personal writing habits. Furthermore, the concept can be extended furthermore to recognize the traffic sign while driving [16].…”

Visualizing Web Images Using Fisher Discriminant Locality Preserving Canonical Correlation Analysis

“…It has been reported in [27] and [28] that a scheme that utilizes several kinds of features provides better performance for multimedia analysis than does a scheme that utilizes only one feature. However, since the above dimensionality reduction methods can utilize only one kind of feature, it is difficult for these methods to use several kinds of modalities effectively when they are applied to multimedia data.…”

Socializing the Semantic Gap

“…One, leverage images visually close to the test image [Li et al 2009bVerbeek et al 2010;Wu et al 2011;Feng et al 2012]. Two, exploit relationships between images labeled with the same tag [Liu et al 2009;Richter et al 2012;Liu et al 2011b;Kuo et al 2012;Gao et al 2013]. Three, learn visual classifiers from socially tagged examples Chen et al 2012;Yang et al 2014].…”