Visually weighted neighbor voting for image tag relevance learning

Abstract

The presence of non-relevant tags in image folksonomies hampers the effective organization and retrieval of user-contributed images. In this paper, we propose to learn the relevance of user-supplied tags by means of visually weighted neighbor voting, a variant of the popular baseline neighbor voting algorithm proposed by Xirong Li et al. in 2009. To gain insight into the effectiveness of baseline and visually weighted neighbor voting, we qualitatively analyze the difference in tag relevance when using a different number of neighbors, for both tags relevant and tags not relevant to the content of a seed image. Our qualitative analysis shows that tag relevance values computed by means of visually weighted neighbor voting are more stable and representative than tag relevance values computed by means of baseline neighbor voting. This is quantitatively confirmed through extensive experimentation with MIRFLICKR-25000, studying the variation of tag relevance values as a function of the number of neighbors used (for both tags relevant and tags not relevant with respect to the content of a seed image), as well as the influence of tag relevance learning on the effectiveness of image tag refinement, tag-based image retrieval, and image tag recommendation.

An Integrated Methods for Refined Facial-Image Retrieval using Sift and Click-Through Data

“…Images are retrieved based on tags and their re ranking [1] [2] and [3] is done by relevance score by considering the votes. Keeping this as a basis, Lee and N eve [6] proposed a new method of calculating relevance score by neighbor voting.…”

Social Image Tag Relevance Learning based on Pixel Voting

“…In this section, six algorithms are used for contrast experiments. Three of them are baseline methods including neighbor voting [20], weighted neighbor voting [3] and rank based neighbor voting [19]. And another three Pixel Voting algorithm are based different distributes.…”

“…According to [1,2], there are only 10% images having their most relevant tags at the first place. Therefore, more and more meaningful and challenge work based tag relevance learning algorithms is attracting people's attention, such as tag de-noising [3][4][5], tag recommendation [6][7][8] and tag ranking [9][10][11],which are studied to improve social images retrieval efficiency.…”

Socializing the Semantic Gap

“…These methods are nonparametric, and the complexity of the learned hypotheses grows as the amount of training data increases. The neighbor voting algorithm [Li et al 2009b] and its variants [Kennedy et al 2009;Truong et al 2012;Lee et al 2013;Zhu et al 2014] estimate the relevance of a tag t with respect to an image x by counting the occurrence of t in annotations of the visual neighbors of x. The visual neighborhood is created using features obtained from early fusion of global features [Li et al 2009b], distance metric learning to combine local and global features [Verbeek et al 2010;Wu et al 2011], cross-modal learning of tags and image features [Qi et al 2012;Ballan et al 2014;Pereira et al 2014], and fusion of multiple single-feature learners Li 2016].…”

“…While the standard neighbor voting algorithm [Li et al 2009b] simply lets the neighbors vote equally, efforts have been made to (heuristically) weight neighbors in terms of their importance. For instance, in Truong et al [2012] and Lee et al [2013], the visual similarity is used as the weights. As an alternative to such a heuristic strategy, Zhu et al [2014] model the relationships among the neighbors by constructing a directed voting graph, wherein there is a directed edge from image x i to image x j if x i is in the k nearest neighbors of x j .…”