World-scale mining of objects and events from community photo collections

Abstract

In this paper, we describe an approach for mining images of objects (such as touristic sights) from community photo collections in an unsupervised fashion. Our approach relies on retrieving geotagged photos from those web-sites using a grid of geospatial tiles. The downloaded photos are clustered into potentially interesting entities through a processing pipeline of several modalities, including visual, textual and spatial proximity. The resulting clusters are analyzed and are automatically classified into objects and events. Using mining techniques, we then find text labels for these clusters, which are used to again assign each cluster to a corresponding Wikipedia article in a fully unsupervised manner. A final verification step uses the contents (including images) from the selected Wikipedia article to verify the cluster-article assignment. We demonstrate this approach on several urban areas, densely covering an area of over 700 square kilometers and mining over 200,000 photos, making it probably the largest experiment of its kind to date.

Extracting Emergent Semantics from Large-Scale User-Generated Content

“…In [10,11] tags and visual information together with geo-location are used for object (e.g. monuments) and event extraction.…”

“…However, more advanced techniques and applications [9,10,13,22,23,24,25] have also been presented, capable of processing more kinds of input modalities enabling the spatio-temporal and situational dimension. Scalability is addressed by a wide range of applications; however the amount of works enabling the real-time aspect is still very limited.…”

Geotag propagation in social networks based on user trust model

“…Another application that combines textual and visual techniques has been proposed by Quack et al [20]. They developed a system that crawls photos on the internet and identifies clusters of images referring to a common object (physical items on fixed locations), and events (special social occasions taking place at certain times).…”

“…Gammeter et al [9] extends this idea towards object-based auto-annotation of holiday photos in a large database that includes landmark buildings, statues, scenes, pieces of art, with help of external resources such as Wikipedia. In both [20] and [9], GPS coordinates are used to pre-cluster objects which may not be always available.…”

Tag Suggestr: Automatic Photo Tag Expansion Using Visual Information for Photo Sharing Websites

“…But, though they are still developing, vision based methods are quite powerful and when combined with textual methods, very effective automated systems can be achieved. A good application of combined use of textual and visual techniques is proposed by Quack et al in [11]. Objective of the work persented in [11] is to provide a system that automatically forms high quality image databases using the large-scale internet sources.…”