A scalable algorithm for extraction and clustering of event-related pictures

Abstract

The event detection problem, which is closely related to clustering, has gained a lot of attentions within event detection for textual documents. However, although image clustering is a problem that has been treated extensively in both Content-Based Image Retrieval (CBIR) and Text-Based Image Retrieval (TBIR) systems, event detection within image management is a relatively new area. Having this in mind, we propose a novel approach for event extraction and clustering of images, taking into account textual annotations, time and geographical positions. Our goal is to develop a clustering method based on the fact that an image may belong to an event cluster. Here, we stress the necessity of having an event clustering and cluster extraction algorithm that are both scalable and allow online applications. To achieve this, we extend a well-known clustering algorithm called Suffix Tree Clustering (STC), originally developed to cluster text documents using document snippets. The idea is that we consider an image along with its annotation as a document. Further, we extend it to also include time and geographical position so that we can capture the contextual information from each image during the clustering process. This has appeared to be particularly useful on images gathered from online photo-sharing applications such as Flickr. Hence, our STC-based approach is aimed at dealing with the challenges induced by capturing contextual information from Flickr images and extracting related events. We evaluate our algorithm using different annotated datasets mainly gathered from Flickr. As part of this evaluation we investigate the effects of using different parameters, such as This paper is an extended and revised version of IEEE ISM 2010 [34] Multimed Tools Appl time and space granularities, and compare these effects. In addition, we evaluate the performance of our algorithm with respect to mining events from image collections. Our experimental results clearly demonstrate the effectiveness of our STC-based algorithm in extracting and clustering events.

Foto2Events: From Photos to Event Discovery and Linking in Online Social Networks

“…Recently, numerous studies have investigated the problem of event detection from images. They can be classified in two groups: event clustering approaches [5][6][7][8][9][10][11] and event hybrid approaches [12,14,17,21,27,28]. Extracting events from multimedia in terms of photographs or images is much more difficult when compared to text for essentially two reasons: i) Event detection from images requires aggregation of heterogeneous metadata [29]; ii) Linking multimedia data to event model aspects is far more challenging then textual data [30].…”

“…Following Nayak [39], for an obtained cluster C i in the clustering solution, the entropy of a cluster is defined as: ( 10 ) where k is the number of events generated, and is the number of records of the r th event that are assigned to the i th cluster. The entire clustering solution entropy is the sum of the individual cluster entropies weighted according to the cluster size.…”

“…Many of these approaches are quite interesting and can be classified in two groups: event clustering approaches and event hybrid approaches. Event clustering approaches [5][6][7][8][9][10][11] use spatial, temporal or spatio-temporal information to cluster images. Event hybrid approaches apply specific models [12,13] and methods such as graph-based algorithms [14,15].…”

Geo-Temporal Mining and Searching of Events from Web-based Image Collections

“…However, although mining and retrieving pictures related to real-life events is an active field, it is still a less mature research domain [144,55,168]. Most existing related approaches have been aimed at extracting events from different types of datasets.…”

Exploring temporal proximity and spatial distribution of terms in web-based search of event-related images

“…Here, an event has a specific semantic meaning. Focusing on media-sharing applications, an event can be "something happening in a certain place at a certain time and tagged with a certain term" [16]. So in an Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”