What does a Text Classifier Learn about Morality? An Explainable Method for Cross-Domain Comparison of Moral Rhetoric

Abstract

Moral rhetoric influences our judgement. Although social scientists recognize moral expression as domain specific, there are no systematic methods for analyzing whether a text classifier learns the domain-specific expression of moral language or not. We propose Tomea, a method to compare a supervised classifier's representation of moral rhetoric across domains. Tomea enables quantitative and qualitative comparisons of moral rhetoric via an interpretable exploration of similarities and differences across moral concepts and domains. We apply Tomea on moral narratives in thirtyfive thousand tweets from seven domains. We extensively evaluate the method via a crowd study, a series of cross-domain moral classification comparisons, and a qualitative analysis of cross-domain moral expression.

Do Differences in Values Influence Disagreements in Online Discussions?

“…Applications include dialogues about moral scenarios (Qiu et al, 2021), review texts (Obie et al, 2021), and value-laden arguments (Kobbe et al, 2020;Alshomary et al, 2022). However, both the annotation and extraction of values remain difficult, with specific questions relating to the granularity of the value labels (Kiesel et al, 2022), their transfer to new domains , and how classifiers understand morality in language (Liscio et al, 2023). Moreover, large variances exist between the frequency of values across domain (Kennedy et al, 2021), and even the relevance of values differs depending on the domain (Bouman et al, 2018;Liscio et al, 2021).…”

LibertyMFD: A Lexicon to Assess the Moral Foundation of Liberty.

“…More recently, Araque et al [2022] proposed a liberty lexicon generation approach based aligning documents from online news sources with different worldviews. The LibertyMFD was later employed by Araque et al [2023] to fine-tune the approach proposed by Consoli et al [2022] for analysing how the Spanish news cover the female (un)employment topic in terms of sentiment and moral values, as well as how this sentiment evolves over time.…”

“…Domain Specific Insights A recurrent pattern is that the overlapping lexicon outperforms the other lexicons in both in-domain and out-of-domain experiments. To gain more insights on the effect of the social context on the moral nuances a specific lemma may have, we employed the TOMEA approach proposed by Liscio et al [2023]. According to TOMEA, the overlapping lexicon differs the most with respect to the BLM lexicons generated by both the CS and the WE methods with scores .17 and .13, respectively.…”