Mitigating Language-Dependent Ethnic Bias in BERT

Abstract

BERT and other large-scale language models (LMs) contain gender and racial bias. They also exhibit other dimensions of social bias, most of which have not been studied in depth, and some of which vary depending on the language. In this paper, we study ethnic bias and how it varies across languages by analyzing and mitigating ethnic bias in monolingual BERT for English, German, Spanish, Korean, Turkish, and Chinese. To observe and quantify ethnic bias, we develop a novel metric called Categorical Bias score. Then we propose two methods for mitigation; first using a multilingual model, and second using contextual word alignment of two monolingual models. We compare our proposed methods with monolingual BERT and show that these methods effectively alleviate the ethnic bias. Which of the two methods works better depends on the amount of NLP resources available for that language. We additionally experiment with Arabic and Greek to verify that our proposed methods work for a wider variety of languages.

An innovative method to investigate the altering urban thermal environment by dynamic land cover change: A case study of Suwon, Republic of Korea

“…To quantify social bias, binary-category metrics such as LP BS [11] for Gender bias and Racial bias, and multi-category metric CBS [12] are employed, as illustrated in Fig. 2.…”

“…We compare and analyze the changes in bias between the original base model and the finetuned model. For quantifying societal bias, we utilize the Categorical Bias Score (CBS) [12] for multi-category Ethnic bias and the Log-Probability Bias Score (LPBS) [11] for binary-category Gender and Racial biases. To mitigate bias, we employ methods such as adjusting the frequency of specific target words and modifying the existing training data set by transforming attributes from harmful to non-harmful words.…”

“…For instance, adversarial biases emerged due to conflicts such as the U.S.-Afghanistan war, Korea-Japan relations, and the North Korean war, and these negative contexts are reflected in the ethnic biases of English (EN-1) and Korean (KO-1). This signifies a reflection of the historical and societal context of the respective countries [12]. Perceptions that certain occupations are more suitable for either males or females are reflected in the gender biases of English (EN-2) and Korean (KO-2), contributing to the formation of fixed stereotypes about those occupations [13].…”

Beyond Discrimination: Generative AI Applications and Ethical Challenges in Forensic Psychiatry

“…Furthermore, an analysis of GPT-2 and GPT-3.5 revealed a propensity to generate masculine-associated pronouns more frequently than feminine-associated ones and show gender-biased association in the context of professions, considering occupations such as Doctor or Engineer as masculine more often than roles like Nurse and Teacher, often regarded as feminine (81). Language-dependent ethnic biases, involving the over-generalised association of an ethnic group to specific attributes, mostly negative, have been found in BERT, where non-toxic comments are incorrectly labelled as toxic when including Middle Eastern country names (82).…”