Dictionaries and distributions: Combining expert knowledge and large scale textual data content analysis

Abstract

Theory-driven text analysis has made extensive use of psychological concept dictionaries, leading to a wide range of important results. These dictionaries have generally been applied through word count methods which have proven to be both simple and effective. In this paper, we introduce Distributed Dictionary Representations (DDR), a method that applies psychological dictionaries using semantic similarity rather than word counts. This allows for the measurement of the similarity between dictionaries and spans of text ranging from complete documents to individual words. We show how DDR enables dictionary authors to place greater emphasis on construct validity without sacrificing linguistic coverage. We further demonstrate the benefits of DDR on two real-world tasks and finally conduct an extensive study of the interaction between dictionary size and task performance. These studies allow us to examine how DDR and word count methods complement one another as tools for applying concept dictionaries and where each is best applied. Finally, we provide references to tools and resources to make this method both available and accessible to a broad psychological audience. Keywords Methodological innovation · Text analysis · Semantic representation · Dictionary-based text analysisElectronic supplementary material The online version of this article

MoralStrength: Exploiting a moral lexicon and embedding similarity for moral foundations prediction

“…More recently, Garten et al [36] employed the MFD to detect moral rhetoric in general, and more specifically, shifts in long political speeches over time. Then, based on psychological dictionaries and semantic similarity to quantify the presence of moral sentiment around a given topic, Garten et al [37], proposed the Distributed Dictionary Representations (DDR) method. Showing promising results, DDR was also employed by Hoover et al [38] to detect moral values in charitable giving.…”

Moral Framing and Charitable Donation: Integrating Exploratory Social Media Analyses and Confirmatory Experimentation

“…Beyond focusing on an under-explored domain, this research demonstrates how cutting-edge, data-driven Natural Language Processing (NLP) methods can be used with theoretical constraints and integrated into an experimental research paradigm. In Study 1, we estimate the semantic association between charitable donation sentiment and moral values using DDR (Garten et al, 2017), an NLP framework that uses distributed representations (Le & Mikolov, 2014;Mikolov, Yih, & Zweig, 2013) learned by a neural network to measure the presence of latent semantic constructs in short texts. Specifically, we rely on DDR to model the association between expressions of moral values and language associated with charitable donation in a corpus of tweets posted during and after Hurricane Sandy.…”

“…This approach pairs a theoretically constrained exploratory social media study with subsequent confirmatory experimental studies. To generate exploratory hypotheses, we estimate a set of hierarchical linear models using measurements obtained via a recently developed Natural Language Processing algorithm, Distributed Dictionary Representation (DDR; Garten et al, 2017), that harnesses the power of data-driven language modeling but also offers the precision of theory-driven measurement specificity. We then programmatically test these hypotheses with a series of preregistered, confirmatory experiments.…”

CLPsych 2018 Shared Task: Predicting Current and Future Psychological Health from Childhood Essays

“…The word-level embeddings are Word2Vec (Mikolov et al, 2013) embeddings learned from the age 50 essay training set; words that appeared less than ten times were replaced with an out-of-vocabulary token. This approach is similar to that of Garten et al (2017), which uses embeddings to capture semantic similarity when applying psychological lexica. It's also similar in motivation to metrics like TERp (Snover et al, 2009) and ME-TEOR (Denkowski and Lavie, 2014) which leverage semantic similarity for evaluating language generation.…”