Twitter spammer detection using data stream clustering

Miller, Zachary; Dickinson, Brian P.; Deitrick, William; Hu, Wei; Wang, Alex Hai

doi:10.1016/j.ins.2013.11.016

Cited by 270 publications

(165 citation statements)

References 6 publications

Supporting

Mentioning

160

Contrasting

Unclassified

Order By: Relevance

“…This presents a number of different challenges since many of the traditional batch learning methods fail when applied to a data stream since we have a number of limitations which are not faced in batch learning [11]. First, because the stream of tweets is potentially unbounded in size, algorithms in this environment have restrictions on memory usage.…”

Section: Twitter Data As a Data Streammentioning

confidence: 99%

Twitter Sentiment in Data Streams with Perceptron

Aston¹,

Liddle²,

Hu³

2014

JCC

Self Cite

View full text Add to dashboard Cite

With the huge increase in popularity of Twitter in recent years, the ability to draw information regarding public sentiment from Twitter data has become an area of immense interest. Numerous methods of determining the sentiment of tweets, both in general and in regard to a specific topic, have been developed, however most of these functions are in a batch learning environment where instances may be passed over multiple times. Since Twitter data in real world situations are far similar to a stream environment, we proposed several algorithms which classify the sentiment of tweets in a data stream. We were able to determine whether a tweet was subjective or objective with an error rate as low as 0.24 and an F-score as high as 0.85. For the determination of positive or negative sentiment in subjective tweets, an error rate as low as 0.23 and an F-score as high as 0.78 were achieved.

show abstract

Section: Twitter Data As a Data Streammentioning

confidence: 99%

Twitter Sentiment in Data Streams with Perceptron

Aston¹,

Liddle²,

Hu³

2014

JCC

Self Cite

View full text Add to dashboard Cite

show abstract

“…The number of email correctly classified by Bayesian filter is denoted by 1 . is the number of emails classified incorrectly by Bayesian filter.…”

Section: Comparisons With Other Methods and Accuracy Of The Cpsfsmentioning

confidence: 99%

“…However, the abuse of bulk emails allows spam to spread like a plague. Spam consumes network bandwidth and brings also other threats to recipients: unwanted advertisements and pornographic content, as well as malicious viruses [1]. A spammer does not need to get permissions from recipients when sending spam, which causes serious annoyance to people and even leads to information security risks [2].…”

Section: Introductionmentioning

confidence: 99%

CPSFS: A Credible Personalized Spam Filtering Scheme by Crowdsourcing

Liu

Zou

Zhang

et al. 2017

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Email spam consumes a lot of network resources and threatens many systems because of its unwanted or malicious content. Most existing spam filters only target complete-spam but ignore semispam. This paper proposes a novel and comprehensive CPSFS scheme: Credible Personalized Spam Filtering Scheme, which classifies spam into two categories: complete-spam and semispam, and targets filtering both kinds of spam. Complete-spam is always spam for all users; semispam is an email identified as spam by some users and as regular email by other users. Most existing spam filters target complete-spam but ignore semispam. In CPSFS, Bayesian filtering is deployed at email servers to identify complete-spam, while semispam is identified at client side by crowdsourcing. An email user client can distinguish junk from legitimate emails according to spam reports from credible contacts with the similar interests. Social trust and interest similarity between users and their contacts are calculated so that spam reports are more accurately targeted to similar users. The experimental results show that the proposed CPSFS can improve the accuracy rate of distinguishing spam from legitimate emails compared with that of Bayesian filter alone.

show abstract

“…With its enormous volume, Twitter also contains messages from automated agents, whose purpose is to advertise, promote or manage perception about specific topics. In academics, spam detection in Twitter is generally seen as a binary classification problem where a tweet is either a spam or not a spam and the spam detection algorithms originate generally from classification methods [34] [35] [36] [37] [38]. Spam detection aims to either identify automated users by using features such as user creation date, username selection and posting patterns or it aims to detect spam tweet by alienating spams based on content [39].…”

Section: Related Workmentioning

confidence: 99%

Tweets on a tree: Index-based clustering of tweets

Erpam¹

2017

View full text Add to dashboard Cite

Computer-mediated communication, CMC, is a type of communication that occurs through use of two or more electronic devices. With the advancement of technology, CMC has started to become a more preferred type of communication between humans. Through computer-mediated technologies, news portals, search engines and social media platforms such as Facebook, Twitter, Reddit and many other platforms are created. In social media platforms, a user can post and discuss his/her own opinion and also read and share other users' opinions. This generates a significant amount of data which, if filtered and analyzed, can give researchers important insights about public opinion and culture.Twitter is a social networking service founded in 2006 and became widespread throughout the world in a very short time frame. The service has more than 310 million monthly active users and throughout these users more than 500 million tweets are generated daily as of 2016. Due the volume, velocity and variety of Twitter data, it cannot be analyzed by using conventional methods. A clustering or sampling method is necessary to reduce the amount of data for analysis.To cluster documents, in a very broad sense two similarity measures can be used: Lexical similarity and semantic similarity. Lexical similarity looks for syntactic similarity between documents. It is usually computationally light to compute lexical similarity, however for clustering purposes it may not be very accurate as it disregards the semantic value of words. On the other hand, semantic similarity looks for semantic value and relations between words to calculate the similarity and while it is generally more accurate than lexical similarity, it is computationally difficult to calculate semantic similarity.In our work we aim to create computationally light and accurate clustering of short documents which have the characteristics of big data. We propose a hybrid approach of clustering where lexical and semantic similarity is combined together. In our approach, we use string similarity to create clusters and semantic vector representations of words to interactively merge clusters.Keywords: clustering, twitter, summarization, suffix tree, semantic relatedness, data mining $ This technical report is based on the thesis "Tweets on a tree: Index-based clustering of tweets".

show abstract

Twitter spammer detection using data stream clustering

Cited by 270 publications

References 6 publications

Twitter Sentiment in Data Streams with Perceptron

Twitter Sentiment in Data Streams with Perceptron

CPSFS: A Credible Personalized Spam Filtering Scheme by Crowdsourcing

Tweets on a tree: Index-based clustering of tweets

Contact Info

Product

Resources

About