Human values and trust in scientific journals, the mainstream media and fake news

Abstract

What factors influence trust in online information?Americans increasingly get information from social media, public distrust in the mainstream media is growing and "fake news" is an important new phenomenon. This paper examines the factors that influence trust in scientific claims posted via social media, including the use of hyperlinks and readers' values. The paper describes a crowdsourcing-based experimental design using Amazon's Mechanical Turk platform. The core of the experiment was a set of 10 scientific findings reported in open-access, peer-reviewed scientific journals, which were in turn linked to in articles in both the mainstream media and "fake news" sites. Data analysis involved exploration of relationships between trust and the presence or absence of hyperlinks, and between trust and human values using nonparametric statistical methods. In terms of the influence of hyperlinks on trust, inclusion of hyperlinks to scientific journals, mainstream media articles, and even hidden URLs led to higher trust than hyperlinks to "fake news" sites or posts without hyperlinks (p < 0.001). Participants who clicked on hyperlinks to scientific articles placed higher trust in the claims than those who did not (p < 0.001). In terms of the influence of values on trust, values had the most impact in cases where individuals saw, but decided not to click on, hyperlinks; this finding seems to indicate that in the absence of firsthand examination of the hyperlinked sites, participants tend to rely more heavily on their values to determine their trust in a scientific claim. These findings indicate that both the presence and absence of hyperlinks and the values of the reader both significantly impact trust judgments.

Demographic factors and trust in different news sources

“…This paper reports findings from a crowdsourcing-based experimental study of the factors that influence trust in different online news sources. We previously published a preliminary analysis of the first subset of the overall data reported upon in this study; due to space constraints the study design is discussed briefly here, but it is explained in more detail in our prior article (Verma et al, 2017). It is important to note that, in addition to reporting on a larger sample, this paper reports a different facet of the analysis: whereas the previous paper explored clicking behavior and human values, this paper details previously unreported data and analysis about demographic factors.…”

“…We collected data in two phases separated by a period of approximately eight weeks. In the first phase, we collected data using 100 HITs (that dataset was reported in Verma et al, 2017), and used an additional 150 HITs in the second phase for a total of 250 HITs. Within and across both phases, we verified the responses to ensure that none of the workers submitted more than one HIT.…”

“…Each post in a timeline consisted of one scientific claim, and one of five hyperlink conditions: the control condition (no hyperlink), a hyperlink to an open-access scientific journal article which the scientific claim was based on, a hidden hyperlink that redirected to the same scientific journal article (Google, n.d.), a hyperlink to a mainstream media site (Flaxman, Goel, & Rao, 2016;Silverman, 2016), and a hyperlink to a fake news site (Shao, Ciampaglia, Flammini, & Menczer, 2016). For more details about the choice of scientific claims and news sources, and our operationalization of fake news, please see Verma et al (2017). The five conditions mentioned above formed the five levels of a factor we refer to as 'post type' in the rest of this paper.…”

“…Third, based on the design of our study, we were unable to control clicking and browsing behavior, or to observe browsing. While we did measure and analyze the clicking behavior (Verma et al, 2017), this is a weak proxy for browsing behavior, as clicking on a link does not guarantee actual reading of the linked page. This shortcoming is particularly problematic for the hidden hyperlink condition of this study, although it is important to note that it was intended more for the study of clicking behavior than for how trust is influenced by demographic features.…”

“…The goal of this paper is to dig deeper into the role of demographic factors such as gender, age, educational attainment, political leaning, and frequency of social media use on trust in different sources of online science news, including fake news, mainstream media, and scientific journals. The paper reports results from a larger study on factors that influence trust; a previous paper (Verma, Fleischmann, & Koltai, 2017) also reported a different subset of findings from the same study; the prior study reported data on clicking behavior and human values, and did not involve any analysis of demographic factors, the focus of this paper. The findings of this study shed new light on the relative importance of demographic factors in influencing trust, employing an experimental design rather than the more common approach of using surveys; as such, we can measure trust more directly, rather than relying on individuals to accurately assess and report their more abstract trust ratings in broad categories of online news sources.…”