Detecting East Asian Prejudice on Social Media

Abstract

During COVID-19 concerns have heightened about the spread of aggressive and hateful language online, especially hostility directed against East Asia and East Asian people. We report on a new dataset and the creation of a machine learning classifier that categorizes social media posts from Twitter into four classes: Hostility against East Asia, Criticism of East Asia, Meta-discussions of East Asian prejudice, and Non-related. The classifier achieves a macro-F1 score of 0.83. We then conduct an in-depth ground-up error analysis and show that the model struggles with edge cases and ambiguous content. We provide the 20,000 tweet training dataset (annotated by experienced analysts), which also contains several secondary categories and additional flags. We also provide the 40,000 original annotations (before adjudication), the full codebook, annotations for COVID-19 relevance and East Asian relevance and stance for 1,000 hashtags, and the final model.

A Framework for the Computational Linguistic Analysis of Dehumanization

“…While our in-depth case study focuses on one particular social group, this framework can be generalized to study dehumanization across a wide variety of social groups, and this could be a fruitful area of future work. For example, Asians have faced increased prejudice and dehumanization since the beginning of the COVID-19 pandemic (Van Bavel et al, 2020 ; Vidgen et al, 2020 ; Ziems et al, 2020 ). Our framework could be applied to understand who dehumanizes these populations in both news and social media, and how the degree of dehumanization changes over time or varies by region.…”

The Internet and public policy: Future directions

“…For the future, tackling these nebulous but sustained and serious threats will require an “army” of researchers with technological tools the like of which our academic forebears would not have dreamt, in the social sciences at least. For example, research in hate speech requires a marriage of social science and machine learning, with continual need to respond to changing language and targets of hate; an urgent and rapid building of the first machine learning classifier for Sinophobia (Vidgen et al, 2020) was required at the start of the pandemic to tackle the growing problem of hate directed at East Asian people. Control—from regulation to public pressure—over social media platforms will be another enduring topic for Policy and Internet , as sometimes it seems that the internet has fulfilled its early promise to be ungovernable space, as the earliest cyber‐utopians believed (Barlow, 1996), at least not by governments.…”

Findings of the NLP4IF-2021 Shared Tasks on Fighting the COVID-19 Infodemic and Censorship Detection

“…Zhou et al (2020) created the ReCOVery dataset, which combines 2,000 news articles about COVID-19, annotated for their factuality, with 140,820 tweets. Vidgen et al (2020) studied COVID-19 prejudices using a manually labeled dataset of 20K tweets with the following labels: hostile, criticism, prejudice, and neutral. Song et al (2021) collected a dataset of false and misleading claims about COVID-19 from IFCN Poynter, which they manually annotated with the following ten disinformation-related categories:…”