Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biases

Walter, Tobias; Kirschner, Celina; Eger, Steffen; Glavaš, Goran; Lauscher, Anne; Ponzetto, Simone Paolo

doi:10.1109/jcdl52503.2021.00017

Cited by 8 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, an equivalent to human test-takers (e.g., whose gender stereotypes would be assessed with a psychological questionnaire) has to be chosen and there are multiple possible candidates (e.g., a fine-tuned model that is 3. Tools developed for such assessments of training data might be useful for computational social scientists as well (see e.g., Garg et al, 2018;Prystawski et al, 2022;Walter et al, 2021, ). 4.…”

Section: The Translationmentioning

confidence: 99%

Undesirable Biases in NLP: Addressing Challenges of Measurement

Van der Wal,

Bachmann,

Leidinger

et al. 2024

jair

View full text Add to dashboard Cite

As Large Language Models and Natural Language Processing (NLP) technology rapidly develop and spread into daily life, it becomes crucial to anticipate how their use could harm people. One problem that has received a lot of attention in recent years is that this technology has displayed harmful biases, from generating derogatory stereotypes to producing disparate outcomes for different social groups. Although a lot of effort has been invested in assessing and mitigating these biases, our methods of measuring the biases of NLP models have serious problems and it is often unclear what they actually measure. In this paper, we provide an interdisciplinary approach to discussing the issue of NLP model bias by adopting the lens of psychometrics — a field specialized in the measurement of concepts like bias that are not directly observable. In particular, we will explore two central notions from psychometrics, the construct validity and the reliability of measurement tools, and discuss how they can be applied in the context of measuring model bias. Our goal is to provide NLP practitioners with methodological tools for designing better bias measures, and to inspire them more generally to explore tools from psychometrics when working on bias measurement tools. This article appears in the AI & Society track.

show abstract

Section: The Translationmentioning

confidence: 99%

Undesirable Biases in NLP: Addressing Challenges of Measurement

Van der Wal,

Bachmann,

Leidinger

et al. 2024

jair

View full text Add to dashboard Cite

show abstract

“…[46] used a model from social psychology to interpret stereotypical language in texts, mapping words in a twodimensional vector space spanned by the dimensions of "warmth" and "competence" . Reference [47] provided a framework to suggest annotation labels to social science students with which to better annotate social media posts for the task of whether the tweets support or refute policy measures for COVID-19. Reference [48] studied the evolution of social biases (antisemitism and anti-communism) over time in German parliamentary proceedings using word embeddings.…”

Section: Related Workmentioning

confidence: 99%

Measuring Social Solidarity During Crisis: The Role of Design Choices

2022

Self Cite

View full text Add to dashboard Cite

Building on our previous work, we assess how social solidarity towards migrants and refugees has changed before and after the onset of the COVID-19 pandemic, by collecting and analyzing a large, novel, and longitudinal dataset of migration-related tweets. To this end, we first annotate above 2000 tweets for (anti-) solidarity expressions towards immigrants, utilizing two annotation approaches (experts vs. crowds).On these annotations, we train a BERT model with multiple data augmentation strategies, which performs close to the human upper bound. We use this high-quality model to automatically label over 240 000 tweets between September 2019 and June 2021. We then assess the automatically labeled data for how statements related to migrant (anti-)solidarity developed over time, before and during the COVID-19 crisis. Our findings show that migrant solidarity became increasingly salient and contested during the early stages of the pandemic but declined in importance since late 2020, with tweet numbers falling slightly below prepandemic levels in summer 2021. During the same period, the share of anti-solidarity tweets increased in a subsample of COVID-19-related tweets. These findings highlight the importance of long-term observation, preand post-crisis comparison, and sampling in research interested in crisis related effects. As one of our main contributions, we outline potential pitfalls of an analysis of social solidarity trends: for example, the ratio of solidarity and anti-solidarity statements depends on the sampling design, i.e., tweet language, Twitter-user accounts' national identification (country known or unknown) and selection of relevant tweets. In our sample, the share of anti-solidarity tweets is higher in native (German) language tweets and among "anonymous" Twitter users writing in German compared to English-language tweets of users located in Germany.

show abstract

“…Kutuzov et al (2018) found that, as is the case often, most of the existing work focuses on the English language. One notable recent exception from that is the work by Walter et al (2021), in which the authors analysed a corpus of German parliamentary proceedings spanning from 1867 to 2020. They were, for example, able to show an increase in antisemitic rhetoric in the years leading to the seizure of power by the national socialists.…”

Section: Related Workmentioning

confidence: 99%

Tracking Semantic Shifts in German Court Decisions with Diachronic Word Embeddings

Braun

2022

Proceedings of the Natural Legal Language Processing Workshop 2022

View full text Add to dashboard Cite

Language and its usage change over time. While legal language is arguably more stable than everyday language, it is still subject to change. Sometimes it changes gradually and slowly, sometimes almost instantaneously, for example through legislative changes. This paper presents an application of diachronic word embeddings to track changes in the usage of language by German courts triggered by changing legislation, based on a corpus of more than 200,000 documents. The results show the swift and lasting effect that changes in legislation can have on the usage of language by courts and suggest that using time-restricted word embedding models could be beneficial for downstream NLP tasks.

show abstract

Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biases

Cited by 8 publications

References 25 publications

Undesirable Biases in NLP: Addressing Challenges of Measurement

Undesirable Biases in NLP: Addressing Challenges of Measurement

Measuring Social Solidarity During Crisis: The Role of Design Choices

Tracking Semantic Shifts in German Court Decisions with Diachronic Word Embeddings

Contact Info

Product

Resources

About