Application of the principal component analysis to detect semantic differences during the content analysis of social networks

Abstract

In this paper, we propose an approach to semantic differences detection in texts presented in the form of frequency dictionaries. The original text data has been obtained by collecting records on various online communities. We have implemented a specialized software module that allows us to analyze and download both posts and comments from the social network VK's open communities. To build our frequency dictionary, we have developed an algorithm that takes into account the peculiarities of the data collected from social networks. In the article, we propose an approach based on the use of methods reducing the dimension of feature spaces to identify keywords based on the analysis of their frequency of usage. The algorithm we present uses the principal component analysis technique. As a result, we have shown that by using the coefficients of the obtained linear transformation, it is possible to estimate the importance of words. With the help of these estimates, we were able to identify not only key words, but also semantic differences in social networks communities. The proposed approach can also be used to form metrics and calculate the social distance between Internet communities.

Determining the proximity of groups in social networks based on text analysis using big data

“…To determine the proximity of groups, several metrics for the comparison of word indexes were considered: Euclidean distance, city-block distance and Mahalanobis distance [7,8]. The Euclidean distance was chosen, since it is most suitable for this experiment according to the following criteria:…”