Event identification for local areas using social media streaming data

Abstract

Unprecedented success and active usage of social media services result in massive amounts of user-generated data. An increasing interest in the contained information from social media data leads to more and more sophisticated analysis and visualization applications. Because of the fast pace and distribution of news in social media data it is an appropriate source to identify events in the data and directly display their occurrence to analysts or other users. This paper presents a method for event identification in local areas using the Twitter data stream. We implement and use a combined log-likelihood ratio approach for the geographic and time dimension of real-life Twitter data in predefined areas of the world to detect events occurring in the message contents. We present a case study with two interesting scenarios to show the usefulness of our approach.

Editorial: Survey and Experimental Analysis of Event Detection Techniques for Twitter

“…The first technique, log-likelihood ratio (LLH), is a reimplementation of Weiler et al [43], which is realized as a LLH user-defined function that is applied to the grouped set of terms of a time window. In contrast to the original technique that detected events for pre-defined geographical areas, we adjusted the approach to calculate the log-likelihood measure for the frequency of all distinct terms in the current time window against their frequency in the past time windows.…”

“…The second technique, Shifty, is a reimplementation of Weiler et al [47]. In contrast to the original paper, which additionally considers bigrams, we now only use single terms in the analysis.…”

Run-Time and Task-Based Performance of Event Detection Techniques for Twitter

“…Our own Shifty [20] technique calculates a measure that is based on the shift of IDF values of single terms in pairs of successive sliding windows of a pre-defined size. First, the IDF value of each term in a single window (with size s input ) is continuously computed and compared to the average IDF value of all terms within that window.…”

Challenges of event detection from social media streams

“…In addition, the critical factor is the dependency between these selected features since more than one event may be represented by an identical set of features leading to ambiguity [48]. Moreover, these features are used to differentiate between the events within the same topic, since the variation between these events may be relatively minor [49]. In practice, these features might be either content-based features (e.g., TF-IDF scores, emoticons, number of tags) or non-textual features, called meta-data (e.g., number of comments or friends (Facebook), or number of followers (Twitter)).…”