An analysis of 14 Million tweets on hashtag‐oriented spamming*

Sedhai, Surendra; Sun, Aixin

doi:10.1002/asi.23836

Cited by 45 publications

(64 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We combine MinHash clustering and incremental clustering to group tweets collected by keywords TC within a time window into clusters. It has been reported that the MinHash algorithm is effective in grouping near-duplicate tweets (Sedhai & Sun, 2015). However, tweets with a different minimum hash value could also be similar.…”

Section: Event-related Tweets Identificationmentioning

confidence: 99%

Collecting event‐related tweets from twitter stream

Zheng

Sun

2018

Asso for Info Science & Tech

Self Cite

View full text Add to dashboard Cite

Twitter provides a channel of collecting and publishing instant information on major events like natural disasters. However, information flow on Twitter is of great volume. For a specific event, messages collected from the Twitter Stream based on either location constraint or predefined keywords would contain a lot of noise. In this article, we propose a method to achieve both high‐precision and high‐recall in collecting event‐related tweets. Our method involves an automatic keyword generation component, and an event‐related tweet identification component. For keyword generation, we consider three properties of candidate keywords, namely relevance, coverage, and evolvement. The keyword updating mechanism enables our method to track the main topics of tweets along event development. To minimize annotation effort in identifying event‐related tweets, we adopt active learning and incorporate multiple‐instance learning which assigns labels to bags instead of instances (that is, individual tweets). Through experiments on two real‐world events, we demonstrate the superiority of our method against state‐of‐the‐art alternatives.

show abstract

Section: Event-related Tweets Identificationmentioning

confidence: 99%

Collecting event‐related tweets from twitter stream

Zheng

Sun

2018

Asso for Info Science & Tech

Self Cite

View full text Add to dashboard Cite

show abstract

“…Features used by machine learning methods for Twitter spam detection are varying and differ in terms of their level (e.g account, tweet, and campaign), formulations, powerfulness, ease of manipulation, and their suitability for real-time detection. Authors in Sedhai and Sun (2017a) and Yang, Harkreader, and Gu (2013) provide a comprehensive study and analysis regarding hashtag, tweet, account, graph, and timing features with respect to their performance in Twitter spam detection.…”

Section: Machine Learning Approachmentioning

confidence: 99%

“…The most challenging task in creating a large dataset is the annotation process. Currently, researchers are using four ways to generate ground truth, including: manual inspection, blacklists, suspended accounts, and clustering Hu et al, 2014;Hu et al, 2013;Sedhai & Sun, 2017a;Wu, Liu et al, 2017 ). Manual inspection is costly, time-consuming, and some times subjective.…”

Section: Dataset Description and Ground Truthmentioning

confidence: 99%

Unsupervised collective-based framework for dynamic retraining of supervised real-time spam tweets detection model

Washha

Qaroush

Mezghani

et al. 2019

Expert Systems with Applications

View full text Add to dashboard Cite

Spam Social spammersTwitter stream a b s t r a c t Twitter is one of the most popular social platforms. It has changed the way of communication and in-formation dissemination through its real-time messaging mechanism. Recently, it has been used by re-searchers and industries as a new source of data for various intelligent systems, such as tweet sentiment analysis and recommendation systems, which require high data quality. However, due to its flexibility and popularity, Twitter has become the main target for spamming activities such as phishing legitimate users or spreading malicious software, which introduces new security issues and waste resources. There-fore, researchers have developed various machine-learning algorithms to reveal Twitter spam. However, as spammers have become smarter and more crafty, the characteristics of the spam tweets are varying over time making these methods inefficient to detect new spammers tricks and strategies. In addition, some of the employed methods (e.g. blacklisting) or spammer features (e.g. graph-based features) are extremely time-consuming, which hinders the ability to detect spammer activities in real-time. In this paper, we introduce a framework to deal with the volatility of the spam contents and new spamming patterns, called the spam drift. The framework combines the strength of unsupervised machine learning approach, which learns from unlabeled tweets, to retrain a real-time supervised tweet-level spam detec-tion model in a batch mode. A set of experiments on a largescale data set show the effectiveness of the proposed online unsupervised method in adaptively discovers and learns the patterns of new spam activities and achieve stable recall values reaching more than 95%. Although the average spam precision of our method is around 60%, the high spam recall values show the ability of our proposed method in reducing spam drift problems compared to traditional machine learning algorithms.

show abstract

“…One possible reason for tri-gram features not in the list may be due to the sparsity of the tri-gram vocabulary in the dataset. It is interesting to note that most of the top words based on Gini-impurity score are the same as the list of hashtags having the highest spammy-index reported in [17]. (1) FoT having more than 2 hashtags (5) Contains spammy hashtag (2) Hashtag per tweet (12) Contains categorical hashtag (7) FoT having spammy hashtag (14) Fraction of capitalized tweets (2) Contains URL (11) FoT that are retweet (6) Is retweet…”

Section: Feature Analysismentioning

confidence: 99%

Semi-Supervised Spam Detection in Twitter Stream

Sedhai

Sun

2018

IEEE Trans. Comput. Soc. Syst.

Self Cite

124

View full text Add to dashboard Cite

Most existing techniques for spam detection on Twitter aim to identify and block users who post spam tweets. In this paper, we propose a Semi-Supervised Spam Detection (S 3 D) framework for spam detection at tweet-level. The proposed framework consists of two main modules: spam detection module operating in real-time mode, and model update module operating in batch mode. The spam detection module consists of four light-weight detectors: (i) blacklisted domain detector to label tweets containing blacklisted URLs, (ii) near-duplicate detector to label tweets that are nearduplicates of confidently pre-labeled tweets, (iii) reliable ham detector to label tweets that are posted by trusted users and that do not contain spammy words, and (iv) multi-classifier based detector labels the remaining tweets. The information required by the detection module are updated in batch mode based on the tweets that are labeled in the previous time window. Experiments on a large scale dataset show that the framework adaptively learns patterns of new spam activities and maintain good accuracy for spam detection in a tweet stream.

show abstract

An analysis of 14 Million tweets on hashtag‐oriented spamming*

Cited by 45 publications

References 45 publications

Collecting event‐related tweets from twitter stream

Collecting event‐related tweets from twitter stream

Unsupervised collective-based framework for dynamic retraining of supervised real-time spam tweets detection model

Semi-Supervised Spam Detection in Twitter Stream

Contact Info

Product

Resources

About