Run-Time and Task-Based Performance of Event Detection Techniques for Twitter

Abstract

Abstract. Twitter's increasing popularity as a source of up to date news and information about current events has spawned a body of research on event detection techniques for social media data streams. Although all proposed approaches provide some evidence as to the quality of the detected events, none relate this task-based performance to their runtime performance in terms of processing speed or data throughput. In particular, neither a quantitative nor a comparative evaluation of these aspects has been performed to date. In this paper, we study the runtime and task-based performance of several state-of-the-art event detection techniques for Twitter. In order to reproducibly compare run-time performance, our approach is based on a general-purpose data stream management system, whereas task-based performance is automatically assessed based on a series of novel measures.

Editorial: Survey and Experimental Analysis of Event Detection Techniques for Twitter

“…With one query, it is possible to obtain a bulk of 100 tweets. 5 Therefore, it is possible to crawl 18 000 tweets per 15-minute window and it would take ∼6666 windows with an estimated total response time of 100 000 minutes (∼1666 hours or ∼69 days) on a single machine to crawl all the contained tweets. As this waiting time is prohibitive in practice, we implemented an alternative crawler that only retrieves the content of the tweets based on their identifiers.…”

“…In order to realize streaming implementations of state-of-theart event detection techniques for Twitter, we use Niagarino 8 [5], a data stream management system developed and maintained by our research group. The main purpose of Niagarino is to serve as an easy-to-use and extensible research platform for streaming applications such as the ones presented in the paper.…”

“…For example, the collection of events gathered by the GDELT Project 12 could be a possible option in a case, where events that are otherwise noteworthy are to be detected. In terms of recall, we crawled the Reuters news archive, but in previous work [5] we have shown that the Bloomberg news archive can also be used as an appropriate ground truth.…”

“…Crafting a gold standard manually in order to use textbook precision and recall measures is painstakingly slow and does therefore not scale to the volumes of data generated by Twitter users. In order to address this requirement, we build on our previous work in this field [5] and, in this paper, propose several scalable measures that can be automatically applied to the results of current and future event detection techniques. The specific contributions of this paper are as follows:…”

Stability Evaluation of Event Detection Techniques for Twitter

“…Fig. 1) in the Niagarino data stream management system [27]. The operators with a dashed frame are the components that are modified in our experiments.…”