Extracting actionable information from Security Forums

Abstract

The goal of this work is to systematically extract information from hacker forums, whose information would be in general described as unstructured: the text of a post is not necessarily following any writing rules. By contrast, many security initiatives and commercial entities are harnessing the readily public information, but they seem to focus on structured sources of information. Here, we focus on the problem of analyzing text content in security forums. A key novelty is that we use user profiles and contextual features along with transfer learning approach and also embedding space to help us identify and refine information that we could not get from security forum with trivial analysis. We collect a wealth of data from 5 different security forums. The contribution of our work is twofold; (a) we develop a method to automatically identify through the forums malicious IP addresses (b) we also propose a systematic method to identify and classify user-specified threads of interest into four categories. We further showcase how this information can inform knowledge extraction from the forums. As the cyberwars are becoming more intense, having early accesses to useful information becomes more imperative to remove the hackers firstmove advantage, and our work is a solid step towards this direction.

RIPEx: Extracting Malicious IP Addresses from Security Forums Using Cross-Forum Learning

“…There is a wealth of information that can be extracted from security forums, which motivates this research direction. Earlier work suggests that there is close to four times more malicious IP addresses in forums compared to established databases of such IP addresses [8]. At the same time, there are tens of thousands of IP addresses in the forums, as we will see later.…”

“…These features capture the behaviour of the author, including frequency of posting, average post length etc. These features were introduced by earlier work [8], with the rationale that profiling the author of a post can help us infer their intention and role and thus, improve the classification.…”

“…Extracting IP addresses from security forums. There two main efforts that focus on IP addresses and security forums [7,8] and neither provides the comprehensive solution that we propose here. The most relevant work [8] does not address the Identification problem, and sidesteps the problem of crossforum training by assuming training data for each forum.…”

“…There two main efforts that focus on IP addresses and security forums [7,8] and neither provides the comprehensive solution that we propose here. The most relevant work [8] does not address the Identification problem, and sidesteps the problem of crossforum training by assuming training data for each forum. The earlier work [7] focuses on the spatiotemporal properties of Canadian IP addresses in forums, but assumes that all identified addresses are suspicious and therefore they did not employ a classification method, which is the focus of our work.…”

A Methodology For Large-Scale Identification of Related Accounts in Underground Forums

“…Sundaresan et al extracted Skype handles from public posts and translated those to their actual IP address to characterize the location of users in underground forums [12]. Gharibshah et al presented a cross-correlation between the IP addresses that users post and the database from VirusTotal to understand and characterize malicious users [36]. Egele et al modeled message characteristics on Social Networks to detect compromised (hacked) accounts [37].…”