A comparative study of the class imbalance problem in Twitter spam detection

Li, Chaoliang; Liu, Shigang

doi:10.1002/cpe.4281

Cited by 41 publications

(20 citation statements)

References 40 publications

(104 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most of the work [17][18][19][20][21][22][23] done in removing class imbalance from data streams is based on ensemble-based techniques. Oversampling based Online Bagging (OOB) and Undersampling based Online Bagging (UOB) [17] can solve the online class imbalance problem for a two-class dataset and are capable of monitoring changes in the class imbalance and adapting to the changing class distribution without outside intervention.…”

Section: Class Imbalancementioning

confidence: 99%

See 1 more Smart Citation

Mining Massive E-Health Data Streams for IoMT Enabled Healthcare Systems

Toor

Usman

Younas

et al. 2020

Sensors

View full text Add to dashboard Cite

With the increasing popularity of the Internet-of-Medical-Things (IoMT) and smart devices, huge volumes of data streams have been generated. This study aims to address the concept drift, which is a major challenge in the processing of voluminous data streams. Concept drift refers to overtime change in data distribution. It may occur in the medical domain, for example the medical sensors measuring for general healthcare or rehabilitation, which may switch their roles for ICU emergency operations when required. Detecting concept drifts becomes trickier when the class distributions in data are skewed, which is often true for medical sensors e-health data. Reactive Drift Detection Method (RDDM) is an efficient method for detecting long concepts. However, RDDM has a high error rate, and it does not handle class imbalance. We propose an Enhanced Reactive Drift Detection Method (ERDDM), which systematically generates strategies to handle concept drift with class imbalance in data streams. We conducted experiments to compare ERDDM with three contemporary techniques in terms of prediction error, drift detection delay, latency, and ability to handle data imbalance. The experimentation was done in Massive Online Analysis (MOA) on 48 synthetic datasets customized to possess the capabilities of data streams. ERDDM can handle abrupt and gradual drifts and performs better than all benchmarks in almost all experiments.

show abstract

Section: Class Imbalancementioning

confidence: 99%

“…Chaoliang et al [19] have done work on the class imbalance problem in the context of Twitter spam detection. Due to class imbalance, the spam detectors have not been able to perform up to their potential.…”

Section: Class Imbalancementioning

confidence: 99%

Mining Massive E-Health Data Streams for IoMT Enabled Healthcare Systems

Toor

Usman

Younas

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Features related to this phenomenon were utilised in training machine learning classifiers. Li and Liu [29] analysed how the effect of unbalance datasets can be mitigated in detection tasks.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Unbalanced datasets often affect the performance of detection systems [29], including ours. To mitigate that, we applied the SMOTE resampling technique [36] to balance the data by upscaling the minority class.…”

Section: Datasetmentioning

confidence: 99%

Detection of spam-posting accounts on Twitter

2018

View full text Add to dashboard Cite

Online Social Media platforms, such as Facebook and Twitter, enable all users, independently of their characteristics, to freely generate and consume huge amounts of data. While this data is being exploited by individuals and organisations to gain competitive advantage, a substantial amount of data is being generated by spam or fake users. One in every 200 social media messages and one in every 21 tweets is estimated to be spam. The rapid growth in the volume of global spam is expected to compromise research works that use social media data, thereby questioning data credibility. Motivated by the need to identify and filter out spam contents in social media data, this study presents a novel approach for distinguishing spam vs. non-spam social media posts and offers more insight into the behaviour of spam users on Twitter. The approach proposes an optimised set of features independent of historical tweets, which are only available for a short time on Twitter. We take into account features related to the users of Twitter, their accounts and their pairwise engagement with each other. We experimentally demonstrate the efficacy and robustness of our approach and compare it to a typical feature set for spam detection in the literature, achieving a significant improvement on performance. In contrast to prior research findings, we observe that an average automated spam account posted at least 12 tweets per day at well defined periods. Our method is suitable for real-time deployment in a social media data collection pipeline as an initial preprocessing strategy to improve the validity of research data.

show abstract

“…Ay Karakuş et al 7 introduced a binary classification study for the Turkish language on movie reviews. In addition, text categorization has multiple applications, such as web page classification, 8,9 spam detection, 10,11 author identification, 12,13 and customer relationship management. 14 TC is such a task to categorize unlabelled natural language texts into a predefined set of thematic classes based on their content.…”

Section: Introductionmentioning

confidence: 99%

An improved term weighting scheme for text classification

Zhong

2019

Concurrency and Computation

View full text Add to dashboard Cite

Summary Text representation is a necessary and primary procedure in performing text classification (TC), which first needs to be obtained through an information‐rich term weighting scheme to achieve higher TC performance. So far, term frequency‐inverse document frequency (TF‐IDF) is the most widely used term weighting scheme, but it suffers from two deficiencies. First, the global weighting factors IDF in TF‐IDF approaches infinity if a certain term does not occur in a text. Second, the IDF is equal to zero if a certain term appears in any text. To offset these drawbacks, we first conduct an in‐depth analysis of the current term weighting schemes, and subsequently, an improved term weighting scheme called term frequency‐inverse exponential frequency (TF‐IEF) and its various variants are proposed. The proposed method replaces IDF with the new global weighting factor IEF to characterize the global weighting factor log‐like IDF in the corpus, which can greatly reduce the effect of feature (term) with high local weighting factor TF in term weighting. As a result, a more representative feature can be generated. We carried out a series of experiments on two commonly used data sets (corpora) utilizing Naïve Bayes and support vector machine classifiers to validate the performance of our proposed schemes. Experimental results explicitly reveal that the proposed term weighting schemes come with better performance than the compared schemes.

show abstract

A comparative study of the class imbalance problem in Twitter spam detection

Cited by 41 publications

References 40 publications

Mining Massive E-Health Data Streams for IoMT Enabled Healthcare Systems

Mining Massive E-Health Data Streams for IoMT Enabled Healthcare Systems

Detection of spam-posting accounts on Twitter

An improved term weighting scheme for text classification

Contact Info

Product

Resources

About