Semantics Derived Automatically from Language Corpora Contain Human-like Moral Choices

Jentzsch, Sophie; Schramowski, Patrick; Rothkopf, Constantin A.; Kersting, Kristian

doi:10.1145/3306618.3314267

Cited by 46 publications

(39 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fairness in NLP. Humans are inherently biased, and many studies have revealed human biases and discrimination in natural language (Garg et al, 2018;Jentzsch et al, 2019). Evidence has, for instance, emerged in biased pre-trained word embeddings and semantics derived from language corpora.…”

Section: Related Workmentioning

confidence: 99%

Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach

Cheng¹,

Mosallanezhad²,

Silva³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

The element of repetition in cyberbullying behavior has directed recent computational studies toward detecting cyberbullying based on a social media session. In contrast to a single text, a session may consist of an initial post and an associated sequence of comments. Yet, emerging efforts to enhance the performance of session-based cyberbullying detection have largely overlooked unintended social biases in existing cyberbullying datasets. For example, a session containing certain demographicidentity terms (e.g., "gay" or "black") is more likely to be classified as an instance of cyberbullying. In this paper, we first show evidence of such bias in models trained on sessions collected from different social media platforms (e.g., Instagram). We then propose a contextaware and model-agnostic debiasing strategy that leverages a reinforcement learning technique, without requiring any extra resources or annotations apart from a pre-defined set of sensitive triggers commonly used for identifying cyberbullying instances. Empirical evaluations show that the proposed strategy can simultaneously alleviate the impacts of the unintended biases and improve the detection performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach

Cheng¹,

Mosallanezhad²,

Silva³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…Gonen and Goldberg [37] applied it to the Gender Neutral Version of GloVe called GN-GloVe from Zhao et al [32]. Jentzsch et al [43] uses it with a skip-gram network in the context of Question Answering and Decision Making and [44] creates an algorithm to discover offensive association related with gender, race and other attributes, generating WEAT tests for them. They called this technique Unsupervised Bias Enumeration (UBE).…”

Section: Association Testsmentioning

confidence: 99%

A Survey on Bias in Deep NLP

Garrido-Muñoz¹,

Montejo-Ráez²,

Martínez-Santiago³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep neural networks are hegemonic approaches to many machine learning areas, including natural language processing (NLP). Thanks to the availability of large corpora collections and the capability of deep architectures to shape internal language mechanisms in self-supervised learning processes (also known as "pre-training"), versatile and performing models are released continuously for every new network design. But these networks, somehow, learn a probability distribution of words and relations across the training collection used, inheriting the potential flaws, inconsistencies and biases contained in such a collection. As pre-trained models have found to be very useful approaches to transfer learning, dealing with bias has become a relevant issue in this new scenario. We introduce bias in a formal way and explore how it has been treated in several networks, in terms of detection and correction. Also, available resources are identified and a strategy to deal with bias in deep NLP is proposed.

show abstract

“…Concerning the complexity and intangibility of ethics and morals, we restrict ourselves, as in our previous work (Jentzsch et al, 2019 ), to a rather basic implementation of this construct, following the theories of deontological ethics. These ask which choices are morally required, forbidden, or permitted instead of asking which kind of a person we should be or which consequences of our actions are to be preferred.…”

Section: Assumptions and Backgroundmentioning

confidence: 99%

“…We start by presenting our previous findings (Jentzsch et al, 2019 ) with focusing on quantifing deontological ethics, i.e., finding out, whether an action itself is right or wrong. Following Kim and Hooker ( 2018 ), for the replication we first focus our attention to atomic actions instead of complex behavioral patterns.…”

Section: Introductionmentioning

confidence: 99%

“…This paper is an extension of the conference paper (Jentzsch et al, 2019 ), where we introduced the basic Moral Choice Machine (MCM). Based on extending Caliskan et al's and similar results, we show that standard machine learning can learn not only stereotyped biases but also answers to ethical choices from textual data that reflect everyday human culture.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Moral Choice Machine

Schramowski

Turan

Jentzsch

et al. 2020

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

Allowing machines to choose whether to kill humans would be devastating for world peace and security. But how do we equip machines with the ability to learn ethical or even moral choices? In this study, we show that applying machine learning to human texts can extract deontological ethical reasoning about “right” and “wrong” conduct. We create a template list of prompts and responses, such as “Should I [action]?”, “Is it okay to [action]?”, etc. with corresponding answers of “Yes/no, I should (not).” and "Yes/no, it is (not)." The model's bias score is the difference between the model's score of the positive response (“Yes, I should”) and that of the negative response (“No, I should not”). For a given choice, the model's overall bias score is the mean of the bias scores of all question/answer templates paired with that choice. Specifically, the resulting model, called the Moral Choice Machine (MCM), calculates the bias score on a sentence level using embeddings of the Universal Sentence Encoder since the moral value of an action to be taken depends on its context. It is objectionable to kill living beings, but it is fine to kill time. It is essential to eat, yet one might not eat dirt. It is important to spread information, yet one should not spread misinformation. Our results indicate that text corpora contain recoverable and accurate imprints of our social, ethical and moral choices, even with context information. Actually, training the Moral Choice Machine on different temporal news and book corpora from the year 1510 to 2008/2009 demonstrate the evolution of moral and ethical choices over different time periods for both atomic actions and actions with context information. By training it on different cultural sources such as the Bible and the constitution of different countries, the dynamics of moral choices in culture, including technology are revealed. That is the fact that moral biases can be extracted, quantified, tracked, and compared across cultures and over time.

show abstract

Semantics Derived Automatically from Language Corpora Contain Human-like Moral Choices

Cited by 46 publications

References 12 publications

Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach

Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach

A Survey on Bias in Deep NLP

The Moral Choice Machine

Contact Info

Product

Resources

About