Language Models in Sociological Research: An Application to Classifying Large Administrative Data and Measuring Religiosity

Jensen, Jeffrey L.; Karell, Daniel; Tanigawa-Lau, Cole; Habash, Nizar; Oudah, Mai; Fani, Dhia Fairus Shofia

doi:10.1177/00811750211053370

Cited by 13 publications

(14 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For our second baseline, we follow Jensen et al (2021) who train language models to infer religiosity from Indonesian names. A language model computes the probabilities of n-grams from a training corpus.…”

Section: Baseline: Name2communitymentioning

confidence: 99%

It’s All in the Name: A Character-Based Approach to Infer Religion

Chaturvedi¹,

Chaturvedi²

2023

Polit. Anal.

View full text Add to dashboard Cite

Large-scale microdata on group identity are critical for studies on identity politics and violence but remain largely unavailable for developing countries. We use personal names to infer religion in South Asia—where religion is a salient social division, and yet, disaggregated data on it are scarce. Existing work predicts religion using a dictionary-based method and, therefore, cannot classify unseen names. We provide character-based machine-learning models that can classify unseen names too with high accuracy. Our models are also much faster and, hence, scalable to large datasets. We explain the classification decisions of one of our models using the layer-wise relevance propagation technique. The character patterns learned by the classifier are rooted in the linguistic origins of names. We apply these to infer the religion of electoral candidates using historical data on Indian elections and observe a trend of declining Muslim representation. Our approach can be used to detect identity groups across the world for whom the underlying names might have different linguistic roots.

show abstract

Section: Baseline: Name2communitymentioning

confidence: 99%

It’s All in the Name: A Character-Based Approach to Infer Religion

Chaturvedi¹,

Chaturvedi²

2023

Polit. Anal.

View full text Add to dashboard Cite

show abstract

“…Language models are designed to solve seemingly simple tasks like the prediction of a masked word in a sentence conditional on observed words surrounding it or the prediction of the next word in a left-to-right sequence (Jensen et al 2021). 6 When trained to perform such tasks on large corpora, the resulting models are able to account for extensive variation in word sequence structures, approximating the semantic content of a natural language.…”

Section: Measuring Political Frames In Textsmentioning

confidence: 99%

Politics as Usual? Measuring Populism, Nationalism, and Authoritarianism in U.S. Presidential Campaigns (1952–2020) with Neural Language Models

Bonikowski

Luo²,

Stuhler³

2022

Sociological Methods & Research

View full text Add to dashboard Cite

Radical-right campaigns commonly employ three discursive elements: anti-elite populism, exclusionary and declinist nationalism, and authoritarianism. Recent scholarship has explored whether these frames have diffused from radical-right to centrist parties in the latter’s effort to compete for the former’s voters. This study instead investigates whether similar frames had been used by mainstream political actors prior to their exploitation by the radical right (in the U.S., Donald Trump’s 2016 and 2020 campaigns). To do so, we identify instances of populism, nationalism (i.e., exclusionary and inclusive definitions of national symbolic boundaries and displays of low and high national pride), and authoritarianism in the speeches of Democratic and Republican presidential nominees between 1952 and 2020. These frames are subtle, infrequent, and polysemic, which makes their measurement difficult. We overcome this by leveraging the affordances of neural language models—in particular, a robustly optimized variant of bidirectional encoder representations from Transformers (RoBERTa) and active learning. As we demonstrate, this approach is more effective for measuring discursive frames than other methods commonly used by social scientists. Our results suggest that what set Donald Trump’s campaign apart from those of mainstream presidential candidates was not the invention of a new form of politics, but the combination of negative evaluations of elites, low national pride, and authoritarianism—all of which had long been present among both parties—with an explicit evocation of exclusionary nationalism, which had been articulated only implicitly by prior presidential nominees. Radical-right discourse—at least at the presidential level in the United States—should therefore be characterized not as a break with the past but as an amplification and creative rearrangement of existing political-cultural tropes.

show abstract

“…Nevertheless, people often name their children in ways that (consciously or unconsciously) signal gender, racial/ethnic, religious, and even class membership. 9,[15][16][17] Other times, they resist these associations by deliberately choosing ambiguous names for their children 15,16 , or by changing their own names to have different associations later in life. The aggregate result of these choices is an imperfect cultural consensus around the gendered, racialized, and other associations of many names.…”

Section: Why Are Demographics Correlated With Names?mentioning

confidence: 99%

“…Conduct inference specific to a population using domain expertise. Jensen et al 17 use their knowledge of the Indonesian regency of Indramayu, where the choice of Javanese, Indonesian (Bahasa), or Arabic names for children is a strong signal of religiosity, to develop a custom name-based religiosity inference model that works well in this setting, but would not translate to many other contexts. This method of combining qualitative and quantitative analysis is the inverse of Nelson's 49 computational grounded theory.…”

Section: Recommendationsmentioning

confidence: 99%

Name-Based Demographic Inference and the Unequal Distribution of Misrecognition

Lockhart¹,

King²,

Munsch³

2022

Preprint

View full text Add to dashboard Cite

Academics and companies increasingly draw on large datasets to understand the social world. Name-based demographic ascription tools are widespread for imputing information like gender and race that are often missing from these large datasets, but these approaches have drawn criticism on ethical, empirical, and theoretical grounds. Employing a survey of all authors listed on articles in sociology, economics, and communications journals in the Web of Science between 2015 and 2020, we compared self-identified demographics with name-based imputations of gender and race/ethnicity for 19,924 scholars across four gender ascription tools (genderize.io, M3-inference, R’s `predictrace` and `gender` packages) and four race/ethnicity ascription tools (ethnicolor’s Florida and North Carolina voter models, and R’s `predictrace` and wru packages). We find substantial inequalities in how these tools misgender and misrecognize the race/ethnicity of authors, distributing erroneous ascriptions unevenly along other demographic traits. Because of the empirical and ethical consequences of these errors, scholars need to be cautious with the use of name-based demographic imputation, particularly when studying subgroups. We recommend five principles for the responsible use of name-based demographic ascription.

show abstract

Language Models in Sociological Research: An Application to Classifying Large Administrative Data and Measuring Religiosity

Cited by 13 publications

References 59 publications

It’s All in the Name: A Character-Based Approach to Infer Religion

It’s All in the Name: A Character-Based Approach to Infer Religion

Politics as Usual? Measuring Populism, Nationalism, and Authoritarianism in U.S. Presidential Campaigns (1952–2020) with Neural Language Models

Name-Based Demographic Inference and the Unequal Distribution of Misrecognition

Contact Info

Product

Resources

About