Cultural Cartography with Word Embeddings

Abstract

<p>Using the presence or frequency of keywords is a classic approach in the formal analysis of text, but has the drawback of glossing over the relationality of word meanings. Word embedding models overcome this problem by constructing a standardized meaning space where words are assigned a location based on relations of similarity to, and difference from, other words based on how they are used in natural language samples. We show how word embeddings can be put to the task of interpretation via two kinds of navigation. First, one can hold terms constant and measure how the embedding space moves around them—much like astronomers measured the changing of celestial bodies with the seasons. Second, one can also hold the embedding space constant and see how documents or authors move relative to it—just as ships use the stars on a given night to determine their location. Using the empirical case of immigration discourse in the United States, we demonstrate the merits of these two broad strategies to advance formal approaches to cultural analysis.</p>

“Do your part: Stay apart”: Collective intentionality and collective (in)action in US governor's COVID-19 press conferences

“…Then, a word embedding model (Skip Gram Model) analyzed the closest words around the identified terms (e.g., we, Marylanders, people) per state, creating standardized values of closeness. Word embedding models are recognized as being particularly well-suited for text analysis focused on meaning ( Nelson 2021 ; Stoltz and Taylor 2021 ). Thus, the composite measure describes a set of words that revolve around context specific plural words that represent the construction of collective intentionality in the press conferences.…”

The extended computational case method: a framework for research design

“…This transformation, which mirrors the distributional hypothesis of language, allows moving problems of semantic similarity from a frequentist ("how often were certain terms used") to an geometric framework ("how close are certain terms with respect to their contexts of sematic use") (Kozlowski et al 2019). The process is 'cartographic' (Stoltz and Taylor 2020), identifying similarities in the use of the tokens in a corpus rather than "actual" meanings. It is, however, powerful for describing how linguistic units evolve over time in their context of use (Bizzoni et al 2019;Hamilton, Leskovec, and Jurafsky 2016;Szymanski 2017) and relation to terms associated to key social cleavages (Garg et al 2018;Rice, Rhodes, and Nteta 2019).…”

All Roads Lead to Polenta: Cultural Attractors at the Junction of Public and Personal Culture

“…We use a word embedding to simulate the actors within the transmission chains. Word embeddings model the meaning of words by representing them as vectors, so that words that appear in semantically similar contexts are close to one another in the embedding space (for sociological adaptation, see Kozlowski et al 2019;Stoltz and Taylor 2020). A common approach to estimating word vectors is the Word2Vec algorithm Mikolov, Sutskever et al 2013), which uses an artificial neural network to learn word vectors by repeatedly (1) taking a passage from the corpus, (2) omitting a word from that passage, (3) attempting to guess the missing word based on the vectors of the context words, (4) assessing the correctness of its guess, and (5) adjusting the word vectors to make future guesses more accurate.…”