KMI-Panlingua-IITKGP @SIGTYP2020: Exploring rules and hybrid systems for automatic prediction of typological features

Abstract

This paper enumerates SigTyP 2020 Shared Task on the prediction of typological features as performed by the KMI-Panlingua-IITKGP team. The task entailed the prediction of missing values in a particular language, provided, the name of the language family, its genus, location (in terms of latitude and longitude coordinates and name of the country where it is spoken) and a set of feature-value pair are available. As part of fulfillment of the aforementioned task, the team submitted 3 kinds of system -2 rule-based and one hybrid system. Of these 3, one rule-based system generated the best performance on the test set. All the systems were 'constrained' in the sense that no additional dataset or information, other than those provided by the organisers, was used for developing the systems.

The Past, Present, and Future of Typological Databases in NLP

“…Past Considering the past years of research, typological databases have mainly been used in the context of feature predictions. Methodologically speaking, features are typically predicted in the context of other features, and other languages (Daumé III and Campbell, 2007;Teh et al, 2009;Berzak et al, 2014;Malaviya et al, 2018;Bjerva et al, 2019cBjerva et al, ,a, 2020Bjerva et al, , 2019bVastl et al, 2020;Jäger, 2020;Choudhary, 2020;Gutkin and Sproat, 2020;Kumar et al, 2020). That is to say, given a language l ∈ L, where L is the set of all languages contained in a specific database, and the features of that language F l , the setup is typically to attempt to predict some subset of features f ⊂ F l , based on the remaining features F l \ f .…”

SIGTYP 2020 Shared Task: Prediction of Typological Features

“…Panlingua (Kumar et al, 2020), a team effort across KMI, Panlingua, and IIT KGP, submitted constrained systems from three approaches: two rule-based systems (one statistical, and one frequency-based baseline) and one hybrid system. Their baseline is similar to the organizers' frequency-base baseline, except that it produces the most frequent value for a feature within a genus if available, backing off to language family, and then the overall most-frequent value.…”

Untitled

“…These remarks may or may not be based on an individual's protected status or protected activities such as race, color, religion, sex, national origin, sexual orientation, or gender identity of an individual [8]. By considering abusive language as an umbrella term, that covers different types of online abuse, extensive studies have been done to address hate speech [3, 8-10, 13, 15, 16], offensive language [1,2,12], cyberbullying [33,34], aggression detection [11,29,34,35], and toxicity detection [36].…”

“…Although many efforts have been dedicated to address the problem of hate speech and offensive language detection in high-resource languages such as English [8,9,50], recently concerns have been raised about other languages as well. Emerging recent shared tasks and academic events such as Kaggle's Toxic Comment Classification Challenge in English, Automatic Misogyny Identification (AMI) at IberEval [17] and EVALITA [4] including Spanish and Italian languages respectively, identification of offensive language at GermEval [2,51] in German language, identification of offensive language at SemEval-2019 [50] for English and SemEval-2020 [1] for Arabic, Danish, English, Greek, and Turkish languages, proceedings of the Workshop on Trolling, Aggression and Cyberbullying Workshops [34,52], and proceedings of the Workshop on Abusive Language Online [5][6][7] shows the raising concerns towards hate speech and offensive language detection in different languages. These events and shared tasks mainly focused on different types of this phenomenon such as hate, offensive, misogyny, aggression, etc.…”

“…Recently great efforts have been taken to investigate the issue of hate speech detection and offensive language identification for different languages in social media; including various competitions such as Kaggle's Toxic Comment Classification Challenge (https://www.kaggle. com/c/jigsaw-toxic-comment-classification-challenge/), Jigsaw Multilingual Toxic Comment Classification (https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification), and conferences and workshops such as SemEval [1], GermEval [2], HatEval [3], EVALITA hate-speech detection task [4], the first [5], second [6], and third [7] editions of the Workshop on Abusive Language Online (https://sites.google.com/view/alw3/), the Second Workshop on Trolling, Aggression and Cyberbullying (https://sites.google.com/view/trac2/home [34]), etc. Furthermore, a great interest has been evidenced in providing annotated corpora in different aspects of offensive language such as Racism and Sexism [8], Hate and Offensive [9], Hate and NoHate [10], Non-aggressive, Overtly-aggressive or Covertly-aggressive [11], Misogynous and Non-misogynous [4], and Not Offensive and Offensive [12].…”