The Evaluative Lexicon 2.0: The measurement of emotionality, extremity, and valence in language

Abstract

The rapid expansion of the Internet and the availability of vast repositories of natural text provide researchers with the immense opportunity to study human reactions, opinions, and behavior on a massive scale. To help researchers take advantage of this new frontier, the present work introduces and validates the Evaluative Lexicon 2.0 (EL 2.0)-a quantitative linguistic tool that specializes in the measurement of the emotionality of individuals' evaluations in text. Specifically, the EL 2.0 utilizes natural language to measure the emotionality, extremity, and valence of evaluative reactions and attitudes. The present article describes how we used a combination of 9 million real-world online reviews and over 1,500 participant judges to construct the EL 2.0 and an additional 5.7 million reviews to validate it. To assess its unique value, the EL 2.0 is compared with two other prominent text analysis tools-LIWC and Warriner et al.'s (Behavior Research Methods, 45, 1191-1207, 2013) wordlist. The EL 2.0 is comparatively distinct in its ability to measure emotionality and explains a significantly greater proportion of the variance in individuals' evaluations. The EL 2.0 can be used with any data that involve speech or writing and provides researchers with the opportunity to capture evaluative reactions both in the laboratory and "in the wild." The EL 2.0 wordlist and normative emotionality, extremity, and valence ratings are freely available from www.evaluativelexicon.com .

The Enhancing Versus Backfiring Effects of Positive Emotion in Consumer Reviews

“… a Although Ludwig et al (2013) used the LIWC’s (Pennebaker et al 2015) positive and negative “emotion” dictionaries, we label this as valence given previous findings that LIWC largely measures valence rather than emotionality (Rocklage, Rucker, and Nordgren 2018a). Notes: We include those papers that focus on emotional content, generally speaking, and that feature products.…”

How online word‐of‐mouth impacts receivers

“…A WOM message may communicate positivity or negativity about product outcomes via star ratings, text content, or both. The causal relationship between these two aspects of valence is bidirectional and nuanced (Ordenes, Ludwig, De Ruyter, Grewal, & Wetzels, ; Rocklage & Fazio, ; Rocklage, Rucker, & Nordgren, , , ). Regardless, reviews with negative star ratings or text can send negative product signals that increase receivers’ subsequent search behavior (Varga & Albuquerque, ) and helpfulness ratings (Cao et al, ), but decrease purchase likelihood (Varga & Albuquerque, ) and brand equity (Bambauer‐Sachse & Mangold, ).…”

“…In a WOM message, emotional content can be conveyed via specific emotion words (e.g., happy, angry), specific non‐emotion words (e.g., disgusting), or other content (e.g., punctuation). The emotionality and the arousal/extremity expressed by particular content can be assessed separately (Rocklage, Rucker, & Nordgren, ): positive or negative words can express similar levels of extremity but different levels of emotionality (e.g., beloved vs. delicious), and can express similar levels of emotionality but different levels of extremity (e.g., perfect vs. lavish). Affective WOM content tends to impact receivers via player signals.…”

“…Little work has simultaneously considered the effects of more than one type of linguistic content—yet exploring the relationships between multiple word categories and different parts of speech is a critical component of building a comprehensive picture of how content impacts receivers. To make this task more manageable, researchers could use various text analysis tools (Berger et al, ; Humphreys & Wang, ; Rocklage et al, ) and rely on broader linguistic frameworks (Pogacar, Shrum, & Lowery, ) or theories (e.g., narrative; Van Laer et al, ) to provide insight into how different aspects of language might combine, and which constellations of words might be impactful.…”

Political ideology shapes the amplification of the accomplishments of disadvantaged vs. advantaged group members

“…We conducted a series of robustness checks to verify our overall conclusions (see SI Appendix for details). In particular, we reconducted all analyses reported above using: (i) only tweets that were identifiably positive [using the Evaluative Lexicon, a computational linguistic approach (24,25)]; (ii) leaveone-out analyses in which we reexamined results excluding each of the 46 Olympians one at a time to ensure that results were not driven by idiosyncratic characteristics of individual athletes; (iii) analyses that tested our models using only a single Olympic event (i.e., swimming, which had the fullest representation of gold medalists across the four demographic categories we consider), to ensure that results were not merely due to liberals or conservatives watching different Olympic events where athletes might tend to be of different demographics; and (iv) analyses that examined the effects of ideology without considering user race or gender, allowing us to use all available tweets (i.e., including tweets from nonblack minorities and those for which user race and/or gender could not be identified). All analyses confirmed the main conclusions.…”