Predicting webpage credibility using linguistic features

Abstract

The article focuses on predicting trustworthiness from textual content of webpages. The recent work Olteanu et al. proposes a number of features (linguistic and social) to apply machine learning methods to recognize trust levels. We demonstrate that this approach can be substantially improved in two ways: by applying machine learning methods to vectors computed using psychosocial and psycholinguistic features and in a high-dimensional bag-of-words paradigm of word occurrences. Following [13], we test the methods in two classification settings, as a 2-class and 3-class scenario, and in a regression setting. In the 3-class scenario, the features compiled by [13] achieve weighted precision of 0.63, while the methods proposed in our paper raise it to 0.66 and 0.70. We also examine coefficients of the models in order to discover words associated with low and high trust.

Belittling the Source: Trustworthiness Indicators to Obfuscate Fake News on the Web

“…In the following, we list some relevant issues encountered while performing our experiments: Experimental results: this gap is also observed w.r.t. results reported by (Olteanu et al, 2013), which is acknowledged by (Wawer et al, 2014), despite numerous attempts to replicate experiments. Authors (Wawer et al, 2014) believe this is due to the lack of parameters and hyperparameters explicitly cited in the previous research (Olteanu et al, 2013).…”

“…Recent research use the website label (Likert scale) released in the Microsoft dataset as a gold standard to train automated web credibility models, as follows: Olteanu et al (2013) proposes a number of properties (37 linguistic and textual features) and applies machine learning methods to recognize trust levels, obtaining 22 relevant features for the task. Wawer et al (2014) improve this work using psychosocial and psycholinguistic features (through The General Inquirer (GI) Lexical Database (Stone and Hunt, 1963)) achieving state of the art results.…”

Reply-Aided Detection of Misinformation via Bayesian Deep Learning

“…In order to reveal linguistic differences between true and false claims, lexical and syntactic features at character, word, sentence and document level have been exploited [1,11,33,36]. Wawer et al [43] compute psycholinguistic features using a bag-of-words paradigm. Rashkin et al [34] compare the language of true claims with that of satire, hoaxes, and propaganda to find linguistic characteristics of untrustworthy text.…”

Modelling Dynamic Interactions Between Relevance Dimensions

“…In recent years, certain criteria or 'dimensions' are widely accepted as the most important considerations for user judgements of relevance. These include reliability [25,34,43,47], understandability [26,51], novelty [7,49], effort [20,40,48], etc. A multidimensional relevance model was proposed [46,50] which defined five such dimensions and was extended to seven dimensions in [22], including 'interest' and 'habit' dimensions.…”