Web Text Categorization Based on Statistical Merging Algorithm in Big Data Environment

Abstract

In the field of modern information technology, how to find information quickly, accurately and comprehensively that users really needed has become the focus of research in this field. In this article, a feature selection method based on a complex network is proposed for the structure and content characteristics of large-scale web text information. The preprocessed web text is converted into a complex network. The nodes in the network correspond to the entries in the text. The edges of the network correspond to the links between the entries in the text, and the degree of nodes and the aggregation system are used. Second, the text classification method is studied from the point of view of data sampling, and a text classification method based on density statistics is proposed. This method uses not only the density information of the text feature set in the classification process, but also the use of statistical merging criteria to get the text. The difference information of each feature has a better classification effect for large text collections.

Feature Selection of Non-Dermoscopic Skin Lesion Images for Nevus and Melanoma Classification

“…The proposed method achieved an accuracy of 92.40% but is computationally expensive. Accordingly, due to the effectiveness of the proposed approach, it is suggested to compare the proposed approach with other feature selection methods for benchmarking and other previous studies on dermoscopic images, such as in [28][29][30][31][32]. In the future, we intend to migrate this method to a mobile application.…”

Evaluating keyphrase extraction algorithms for finding similar news articles using lexical similarity calculation and semantic relatedness measurement by word embedding

“…The supervised approach ( Turney, 2002 ) transforms the keyphrase extraction work into a classification or regression problem ( Wang & Wang, 2019 ). It employs the learned model to identify if a candidate phrase in a text is a keyphrase by training it on the labeled training set.…”

Predicting Violence-Induced Stress in an Arabic Social Media Forum

“…Authors in [30] found numerous aspects in the data collected by victims, such as negative feelings, isolation, and repeated pattern of fear terms. The authors in [31] used behavioral trends on Facebook to predict depression following violence incidence by self-reported victims through their Facebook status. They performed t-tests to distinguish the victims' behaviors in first violence incidence and in the repeated ones.…”

“…These data were used in the learning random forest classifiers to identify violence victims from non-victims. Authors in [29][30][31][32] collected data from Twitter to predict violence incidence from tweeter. Authors in [33], employed analysis dynamics in physician rating websites during the early wave of the COVID-19 pandemic.…”