Detecting Trending Terms in Cybersecurity Forum Discussions

Abstract

We present a lightweight method for identifying currently trending terms in relation to a known prior of terms, using a weighted logodds ratio with an informative prior. We apply this method to a dataset of posts from an English-language underground hacking forum, spanning over ten years of activity, with posts containing misspellings, orthographic variation, acronyms, and slang. Our statistical approach supports analysis of linguistic change and discussion topics over time, without a requirement to train a topic model for each time interval for analysis. We evaluate the approach by comparing the results to TF-IDF using the discounted cumulative gain metric with human annotations, finding our method outperforms TF-IDF on information retrieval.

Untitled

“…There is significant interest surrounding the goal of being able to automate cybersecurity threat detection on social media [19,18,15,11,12,27,14,2]. Twitter, Reddit, and Stackexchange are popular forums from which several previous studies have gathered cybersecurity related documents [19,11,14,15,2,20] for the purpose of training machine learning detection systems and classifiers.…”

“…Identifying cybersecurity discussions in open forums at scale is a topic of great interest for the purpose of mitigating and understanding modern cyber threats [12,19,22]. The challenge is that these discussions are typically quite noisy (i.e., they contain community known synonyms or acronyms or slang) and it is difficult to get labelled data in order to train resilient NLP (natural language processing) topic classifiers.…”

An Enhanced Machine Learning Topic Classification Methodology for Cybersecurity

“…There is significant interest surrounding the goal of being able to automate cybersecurity threat detection on social media [18,17,14,10,11,25,13,2]. Twitter, Reddit, and Stackexchange are popular forums from which several previous studies have gathered cybersecurity related documents [18,10,13,14,2,19] for the purpose of training machine learning detection systems and classifiers.…”

“…There are several different approaches taken with which topic modelling task to use as a signal to detect cybersecurity discussions. Typically the topic classification task is related to training directly on labelled text and then perhaps developing an idea of the more relevant keywords in these discussions [18,11]. Other researchers use sentiment analysis in conjunction with other machine learning models [25,10].…”

“…Identifying cybersecurity discussions in open forums at scale is a topic of great interest for the purpose of mitigating and understanding modern cyber threats [11,18,20]. The challenge is that often these discussions are quite noisy (i.e., they contain community known synonyms or acronyms) and difficult to get labelled data in order to train resilient NLP (natural language processing) topic classifiers.…”