Event Detection in Twitter: A Keyword Volume Approach

Abstract

Event detection using social media streams needs a set of informative features with strong signals that need minimal preprocessing and are highly associated with events of interest. Identifying these informative features as keywords from Twitter is challenging, as people use informal language to express their thoughts and feelings. This informality includes acronyms, misspelled words, synonyms, transliteration and ambiguous terms. In this paper, we propose an efficient method to select the keywords frequently used in Twitter that are mostly associated with events of interest such as protests. The volume of these keywords is tracked in real time to identify the events of interest in a binary classification scheme. We use keywords within wordpairs to capture the context. The proposed method is to binarize vectors of daily counts for each word-pair by applying a spike detection temporal filter, then use the Jaccard metric to measure the similarity of the binary vector for each word-pair with the binary vector describing event occurrence. The top n wordpairs are used as features to classify any day to be an event or non-event day. The selected features are tested using multiple classifiers such as Naive Bayes, SVM, Logistic Regression, KNN and decision trees. They all produced AUC ROC scores up to 0.91 and F1 scores up to 0.79. The experiment is performed using the English language in multiple cities such as Melbourne, Sydney and Brisbane as well as the Indonesian language in Jakarta. The two experiments, comprising different languages and locations, yielded similar results.

Local Event Detection Scheme by Analyzing Relevant Documents in Social Networks

“…The existing schemes such as text-based event detection [6] and geo-tag based event detection [17] were chosen for comparison. We compared precision, recall, and F-measure to show the excellence of the proposed scheme.…”

“…In some studies, extracting keywords from posts is utilized for event detection. The authors in [6] proposed not only detecting events but also predicting events in the near future through a time analysis by using words related to the detected events. It utilizes the Jacard similarity coefficient to discover event-related words.…”

“…Real-time event detection schemes [17,22,23,26,28,30,31] assume situations where information that must be provided to users in real time is critical. General event detection schemes [6,13] do not have a definition of events, but they are currently aimed at detecting various issues occurring online. Local event detection schemes [17,[22][23][24]27,28,30] aim to detect events or issues occurring in a particular region in the real world.…”

“…There are three main schemes to detect events. First, there is a scheme to detect an event using the burstiness of a particular word through a language analysis of a document [6,22]. It is not suitable for a local event detection scheme because it analyzes only words.…”

“…Text-based event detection schemes have been proposed for considering the importance and frequency of keywords by analyzing and extracting meaningful words on hashtags, contents, and comments [5][6][7][8][9][10][11][12]. Text-based event detection schemes utilize term frequencyinverse document frequency (TF-IDF) to extract keywords [5].…”