Object-based tag propagation for semi-automatic annotation of images

Abstract

Over the last few years, social network systems have greatly increased users' involvement in online content creation and annotation. Since such systems usually need to deal with a large amount of multimedia data, it becomes desirable to realize an interactive service that minimizes tedious and timeconsuming manual annotation. In this paper, we propose an interactive online platform that is capable of performing semi-automatic image annotation and tag recommendation for an extensive online database. First, when the user marks a specific object in an image, the system performs an object duplicate detection and returns the search results with images containing similar objects. Then, the annotation of the object can be performed in two ways: (1) In the tag recommendation process, the system recommends tags associated with the object in images of the search results, among which, the user can accept some tags for the object in the given image. (2) In the tag propagation process, when the user enters his/her tag for the object, it is propagated to images in the search results. Different techniques to speed-up the process of indexing and retrieval are presented in this paper and their effectiveness demonstrated through a set of experiments considering various classes of objects.

Class-based tag recommendation and user-based evaluation in online audio clip sharing

“…The number of tags we delete is chosen uniformly at random, with the only constraint of leaving a minimum number of input tags of |Γ I | ≥ 3 so that there is presumably enough information for the recommender systems to provide good recommendations [7]. This constraint also implies that in order to be able to remove at least one tag for each audio clip (|Γ D | ≥ 1), we can only consider for evaluation the audio clips that have at least four tags 12 . After we remove some tags, we run the four tag recommendation methods using Γ I as input and the similarity matrices we computed in the training phase.…”

“…In general, tag recommendations are either based on content analysis of online resources or in the other tags that users introduce during the annotation process. In the case of content-based recommendations, a typical approach consists in, given a resource to be described, defining a neighbourhood of other resources (based on some similarity measure) and then recommending tags that are used to annotate resources in this neighbourhood [12,24]. Another approach is the use of machine learning techniques to learn mappings between tags and content features [15,25,26].…”

A survey of image labelling for computer vision applications

“…Most propagation methods are based on some form of clustering. Ivanov et al (2010) use hierarchical k-means clustering of visual features to detect similar pictures. When the user annotates an object on a new image, the system automatically propagates this annotation to similar objects from the database using a duplicate detector.…”

Analysis of the Impact of a Tag Recommendation System in a Real-World Folksonomy

“…Former systems normally use feature extraction techniques to analyse content resources, and further training of machine learning models that can predict tags based on the extracted features (e.g., Li and Wang 2008, Turnbull et al 2008, Toderici et al 2010. Folksonomy-based systems normally take advantage of tag co-occurrence information in previously annotated resources in order to provide relevant tag recommendations for newly annotated resources (e.g., Sigurbjörnsson and Zwol 2008, Garg and Weber 2008, De Meo et al 2009, Ivanov et al 2010, Font et al 2013b).…”