Automatically generated spam detection based on sentence-level topic information

Abstract

Spammers use a wide range of content generation techniques with low quality pages known as content spam to achieve their goals. We argue that content spam must be tackled using a wide range of content quality features. In this paper, we propose novel sentence-level diversity features based on the probabilistic topic model. We combine them with other content features to build a content spam classifier. Our experiments show that our method outperforms the conventional methods.

Model of Network Topic Detection Based on Web Usage Behaviour Mode Analysis and Mining Technology

“…survey research on the method of topic link detection based on improved information bottleneck theory [10], in this paper, a method of representing text is proposed, which can divide text into several sections of sub-topic features based on the regular pattern of semantic distribution and improve information bottleneck theory, then, the text represented by the attributes is utilized to do topic link detection, the experimental results have shown that this method has a fast convergent rate, and can improve the performance of topic link detection system. Suhara, Yoshihiko and others survey research on the method of information detection based on sentence-level topic [11], in this paper, the text sentence-level diversity features based on the probabilistic topic model is proposed, an information content classifier is also constructed combining features proposed, the experimental results show that this method outperforms the conventional methods. Pang, JB and others survey research on the method of unsupervised web topic detection using a ranked clustering-like pattern across similarity cascades [12], in this paper, a method using a clusteringlike pattern across similarity cascades is investigated from the perspective of similarity diffusion, a topic-restricted similarity diffusion process is also proposed to identify real topic from a large number of candidates efficiently, the experimental results demonstrate that this approach outperforms the state-of-the-art methods on several public data sets, those works are related to author's research direction of network topic detection and application.…”

Exploiting latent content based features for the detection of static SMS spams

“…The rational for using topic modeling is that spams have more unusual topic distributions than non-spam messages. For example, Suhara et al (2013) developed a sentence level LDA to assign topics to sentences for web spam detection. Biro et al, (2009) defined a threshold based on the outputs of LDA to distinguish between spam and non-spam.…”

Approaches for Web Spam Detection

“…In [53] the authors extracted features based on sentence-level topic information. They first created LDA [11] with a ham corpus and apply it to the unseen documents to infer the topic distribution of the sentences.…”