Self-reported willingness to share political news articles in online surveys correlates with actual sharing on Twitter

Abstract

There is an increasing imperative for psychologists and other behavioral scientists to understand how people behave on social media. However, it is often very difficult to execute experimental research on actual social media platforms, or to link survey responses to online behavior in order to perform correlational analyses. Thus, there is a natural desire to use selfreported behavioral intentions in standard survey studies to gain insight into online behavior. But are such hypothetical responses hopelessly disconnected from actual sharing decisions? Or are online survey samples via sources such as Amazon Mechanical Turk (MTurk) so different from the average social media user that the survey responses of one group give little insight into the on-platform behavior of the other? Here we investigate these issues by examining 67 pieces of political news content. We evaluate whether there is a meaningful relationship between (i) the level of sharing (tweets and retweets) of a given piece of content on Twitter, and (ii) the extent to which individuals (total N = 993) in online surveys on MTurk reported being willing to share that same piece of content. We found that the same news headlines that were more likely to be hypothetically shared on MTurk were also shared more frequently by Twitter users, r = .44. For example, across the observed range of MTurk sharing fractions, a 20 percentage point increase in the fraction of MTurk participants who reported being willing to share a news headline on social media was associated with 10x as many actual shares on Twitter. We also found that the correlation between sharing and various features of the headline was similar using both MTurk and Twitter data. These findings suggest that self-reported sharing intentions collected in online surveys are likely to provide some meaningful insight into what content would actually be shared on social media.

Shifting attention to accuracy can reduce misinformation online

“…Participants were randomly assigned to either judge each headline's veracity (Accuracy condition) or indicate if they would consider sharing each headline online (Sharing condition). Although these sharing decisions are hypothetical, research suggests that selfreport sharing decisions of news articles like those used in our study correlate strongly with actual sharing on social media 18 . For full methodological details and explanations of preregistrations for all studies, see Supplementary Information (SI).…”

The Implied Truth Effect: Attaching Warnings to a Subset of Fake News Headlines Increases Perceived Accuracy of Headlines Without Warnings

“…Thus, laboratory-style data provide a useful window into the potential effects of interventions aimed at fighting misinformation. This is particularly true given evidence that self-reported social media sharing intentions are predictive of actual sharing on social media (Mosleh et al 2019). Nonetheless, it is our hope that future work will be able to explore the issues raised here in the context of more typical social media use.…”

“…(response options: no, yes). We note that although these sharing decisions were hypothetical, there is reason to believe that participant responses shed light on actual sharing behavior on social media: across a set of political news headlines, Mosleh et al (2019) find that self-reported sharing intentions collected on Mechanical Turk are strongly correlated with the number of tweets and retweets a given headline actually received on Twitter.…”

Identity concerns drive belief: The impact of partisan identity on the belief and dissemination of true and false news

“…Our main dependent variable was belief in several news stories as well as willingness to share these stories on social media. Recent work suggests that the same news headlines that were more likely to be hypothetically shared are also shared more frequently by actual Twitter users (r = .44; Mosleh, Pennycook, & Rand, 2020). As such, we examine the intent to share these stories on social media.…”

“…This is particularly relevant with regards to intent to share (mis)information. Although prior work has found that intentions to share are highly correlated with actual sharing on social media (Mosleh et al, 2020), future work should investigate actual online sharing behavior (e.g., Brady et al, 2017). Moreover, because our measures are self-reported, we are not in a position to identify where in the processing stream the bias might be occurring.…”