Correlating Self-Report and Trace Data Measures of Incivility: A Proof of Concept

Abstract

This study correlated self-report and trace data measures of political incivility. Specifically, we asked respondents to provide estimates of the degree to which they engage in uncivil political communication online. These estimates were then compared to computational measures of uncivil social media discussion behavior. The results indicated that those who self-disclose uncivil online behavior also tend to generate content on social media that is uncivil as identified by Google’s Perspective application programming interface. Taken as a whole, this work suggests that combining self-report and behavioral trace data may be a fruitful means of developing multimethod measures of complex communication behaviors.

A network analysis of political incivility dimensions

“…In both cases, threatening language is affectively negative in nature and employed for the purposes of designating the target as malignant and, thus, worthy derision. This dimension addresses the interpersonal threat component included in Massaro and Stryker (2012), Santana (2014), and Hopp et al (2018), and the threat to democracy dimension found in Rowe (2015). Consistent with prior renderings of incivility in Coe et al (2014), Gervais’ (2014, 2015b), Sobieraj and Berry (2011), and Stryker, Conway, and Danielson (2014), the persuasive deception dimension refers specifically to purposeful lying or weaponized use of ambiguity.…”

“…Despite its potentially deleterious effects on democratic functioning, scholars have yet to settle upon a consistent definition of political incivility (Hopp et al, 2018; Stryker, Conway, & Danielson, 2014; Su et al, 2018). Building upon a synthesis of prior work, however, it is here suggested that incivility attributes can be segmented into two categories, both of which deal with purposeful normative violations.…”

“…Synthesizing the foregoing, this work suggests that there broadly exist six general categories of textually apparent political incivility that pertain to the violation of speech-related norms: profane language, intentional disrespect, use of threat, persuasive deception, invocation of negative of stereotypes , and gratuitousness . The profane language dimension of incivility refers to the untargeted use of vulgar, crude, or profane language and corresponds with the incivility dimensions identified by Coe et al (2014), Santana (2014), Gervais (2015b), Su et al (2018), and Hopp et al (2018). The intentional disrespect dimension corresponds with the operationalization approaches used in Santana (2014; name-calling and abusive language), Coe et al (2014; name-calling), Gervais (2015a; 2015b; invectives and ridicule), Ng and Detenber (2005; name-calling and personal attacks), and Brooks and Geer (2007; insulting language and name-calling).…”

“…For instance, some scholars (e.g. Coe, Kenski, & Rains, 2014; Hopp, Vargo, Dixon, & Thain, 2018; Santana, 2014; Vargo & Hopp, 2017) have focused on the prevalence of factors such swear words, slurs, and racist/xenophobic commentary. Others (e.g.…”

Integrating Survey Data and Digital Trace Data: Key Issues in Developing an Emerging Field

“…When researchers want to analyze the substance of human communication and behavior, they very quickly face constraints due to the sheer size and unstructured nature of the data, which makes hand-coding unfeasible in most cases (but see Haenschen, 2019). The paper by Hopp, Vargo, Dixon, and Thain (2018) in this issue stands out by applying the Google's Perspective API to classify incivility at a larger scale. The authors demonstrate that there is a considerable congruence between self-reports on online incivility and computational measures.…”