STREAMCUBE: Hierarchical spatio-temporal hashtag clustering for event exploration over the Twitter stream

Abstract

Abstract-What is happening around the world? When and where? Mining the geo-tagged Twitter stream makes it possible to answer the above questions in real-time. Although a single tweet can be short and noisy, proper aggregations of tweets can provide meaningful results. In this paper, we focus on hierarchical spatio-temporal hashtag clustering techniques. Our system has the following features: (1) Exploring events (hashtag clusters) with different space granularity. Users can zoom in and out on maps to find out what is happening in a particular area.(2) Exploring events with different time granularity. Users can choose to see what is happening today or in the past week. (3) Efficient single-pass algorithm for event identification, which provides human-readable hashtag clusters. (4) Efficient event ranking which aims to find burst events and localized events given a particular region and time frame. To support aggregation with different space and time granularity, we propose a data structure called STREAMCUBE, which is an extension of the data cube structure from the database community with spatial and temporal hierarchy. To achieve high scalability, we propose a divide-and-conquer method to construct the STREAMCUBE. To support flexible event ranking with different weights, we proposed a top-k based index. Different efficient methods are used to speed up event similarity computations. Finally, we have conducted extensive experiments on a real twitter data. Experimental results show that our framework can provide meaningful results with high scalability.

GARNET: A holistic system approach for trending queries in microblogs

“…Regarding query types, existing techniques address either frequent or trending items discovery, based on the supported application. Event detection applications [1], [3], [9], [14], [16], [20], [21], [29], however, are more focused on grouping several trending keywords together to report an event rather than focusing on the scalability and performance of retrieving trending keywords. Thus, these applications are orthogonal to our work in a way that GARNET can use one of these techniques in order to report an event based on the returned trending keywords.…”

“…7. Frequent/Trending Items in Microblogs: Event Detection [1], [3], [9], [14], [16], [20], [21], [29]; GeoScope [4]; AFIA [24] keywords at different locations in recent time interval. However, AFIA is only able to answer frequent keywords queries and is not geared towards answering trending queries.…”

Never Abandon Minorities: Exhaustive Extraction of Bursty Phrases on Microblogs Using Set Cover Problem

“…Unit of process Measure of burst (Sayyadi et al, 2009) Noun phrase TF, DF, IDF (O'Connor et al, 2010) Uni-gram, bi-gram, tri-gram Burstiness (Mathioudakis and Koudas, 2010) Uni-gram Burstiness (Weng and Lee, 2011) Uni-gram DF-IDF, H-measure (wavelet analysis) (Metzler et al, 2012) Uni-gram Burstiness (Li et al, 2012) N-gram of any length Deviation from Gaussian distribution (Cui et al, 2012) Hashtag Deviation from Gaussian distribution (Benhardus and Kalita, 2013 (Abdelhaq et al, 2013) Uni-gram Deviation from Gaussian distribution (Schubert et al, 2014) Uni-gram (pair) Deviation from Gaussian distribution (Feng et al, 2015) Hashtag Deviation from Gaussian distribution burst (i.e., the rapid increase of occurrence) of a phrase causes the burst of overlapping incomplete N-grams. For example, if phrase "let it be" bursts, the occurrence of some overlapping N-grams such as "let it", "let it be is", and "it be is" inevitably increases, possibly generating bursty incomplete Ngrams.…”

Exploring a Scalable Solution to Identifying Events in Noisy Twitter Streams

“…They introduced a parameter-free clustering approach called feature-pivot clustering, which attempted to detect and cluster bursty features into hot stories. Similarly, [8] interpreted events as hashtag clusters and propose a hierarchical spatio-temporal clustering of tweets into events.…”