Evaluating Machine Learning Techniques for Detecting Offensive and Hate Speech in South African Tweets

Abstract

In recent times, South Africa has been witnessing insurgence of offensive and hate speech along racial and ethnic dispositions on Twitter. Popular among the South African languages used is English. Although, machine learning has been successfully used to detect offensive and hate speech in several English contexts, the distinctiveness of South African tweets and the similarities among offensive, hate and free speeches require domain-specific English corpus and techniques to detect the offensive and hate speech. Thus, we developed an English corpus from South African tweets and evaluated different machine learning techniques to detect offensive and hate speech. Character n-gram, word n-gram, negative sentiment, syntactic-based features and their hybrid were extracted and analyzed using hyper-parameter optimization, ensemble and multi-tier meta-learning models of support vector machine, logistic regression, random forest, gradient boosting algorithms. The results showed that optimized support vector machine with character ngram performed best in detection of hate speech with true positive rate of 0.894, while optimized gradient boosting with word n-gram performed best in detection of hate speech with true positive rate of 0.867. However, their performances in detection of other threatening classes were poor. Multi-tier meta-learning models achieved the most consistent and balanced classification performance with true positive rates of 0.858 and 0.887 for hate speech and offensive speech, respectively as well as true positive rate of 0.646 for free speech and overall accuracy of 0.671. The error analysis showed that multi-tier meta-learning model could reduce the misclassification error rate of the optimized models by 34.26%. INDEX TERMS Machine learning, South Africa, Twitter, hate speech, offensive speech. OLUWAFEMI ORIOLA received the B.Sc. degree (Hons.

XHate-999: Analyzing and Detecting Abusive Language Across Domains and Languages

“…A non-exhaustive list includes Arabic (Mubarak et al, 2017;Chowdhury et al, 2020), Danish (Sigurbergsson and Derczynski, 2020), German (Jaki and De Smedt, 2019;Wiegand et al, 2018b), Hindi (Saroj and Pal, 2020), Italian (Bosco et al, 2018;Fersini et al, 2018), Polish (Ptaszynski et al, 2019), Portuguese (Fortuna et al, 2019), Dutch (Tulkens et al, 2016), and Slovene (Fišer et al, 2017). There is also some work on specific language variants, like Hindi-English code-switched language (Mathur et al, 2018a;Mathur et al, 2018b) or South African English (Oriola and Kotzé, 2020 assurance, each test example candidate has been manually checked by the authors, and replaced by another sample if it (i) comprises only a single non-indicative word, (ii) it is not written in English, or (iii) it relies on world knowledge which is too specific or geographically localized or on contextual information which hinders proper translation. The final English XHATE-999 test set comprises 600, 300 and 99 instances from WUL, TRAC, and GAO, respectively.…”

A Systematic Literature Review of Different Machine Learning Methods on Hate Speech Detection

“…Paper [19] is about detecting hate speech in South African tweets using machine learning approach. Twitter is the most used social media in South Africa.…”

Towards a Framework for Acquisition and Analysis of Speeches to Identify Suspicious Contents through Machine Learning

“…eir SVMbased classifier obtained 77.6% accuracy. Oriola and Kotze [19] developed an English corpus of South African tweets and applied various machine learning algorithms to detect offensive speech.…”