IIITT at CASE 2021 Task 1: Leveraging Pretrained Language Models for Multilingual Protest Detection

Kalyan, Pawan; Reddy, Duddukunta; Hande, Adeep; Priyadharshini, Ruba; Sakuntharaj, Ratnasingam; Chakravarthi, Bharathi Raja

doi:10.18653/v1/2021.case-1.13

Cited by 3 publications

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…So, the model was fine-tuned for three epochs while maintaining a low learning rate of around 3e-5 to get a BLEU score of 24.2. (Popović, 2015;Snover et al, 2006) Training a model to predict for a low-resourced language was highly challenging due to the absence of prominent pretrained models (Kalyan et al, 2021;Yasaswini et al, 2021;Hande et al, 2021b). However, as an experiment, two models from HuggingFace Transformers 11 , M2M100 (Fan et al, 2020) and Opus-MT from Helsinki NLP (Tiedemann, 2020) were compared.…”

Section: Results and Analysismentioning

confidence: 99%

Attentive fine-tuning of Transformers for Translation of low-resourced languages @LoResMT 2021

Puranik¹,

Hande²,

Priyadharshini³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper reports the Machine Translation (MT) systems submitted by the IIITT team for the English→Marathi and English⇔Irish language pairs LoResMT 2021 shared task. The task focuses on getting exceptional translations for rather low-resourced languages like Irish and Marathi. We fine-tune IndicTrans, a pretrained multilingual NMT model for English→Marathi, using external parallel corpus as input for additional training. We have used a pretrained Helsinki-NLP Opus MT English⇔Irish model for the latter language pair. Our approaches yield relatively promising results on the BLEU metrics. Under the team name IIITT, our systems ranked 1, 1, and 2 in English→Marathi, Irish→English, and English→Irish respectively. The codes for our systems are published 1 .

show abstract

Section: Results and Analysismentioning

confidence: 99%

Attentive fine-tuning of Transformers for Translation of low-resourced languages @LoResMT 2021

Puranik¹,

Hande²,

Priyadharshini³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

TTL: transformer-based two-phase transfer learning for cross-lingual news event detection

Hettiarachchi

Adedoyin-Olowe

Bhogal

et al. 2023

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

Today, we have access to a vast data amount, especially on the internet. Online news agencies play a vital role in this data generation, but most of their data is unstructured, requiring an enormous effort to extract important information. Thus, automated intelligent event detection mechanisms are invaluable to the community. In this research, we focus on identifying event details at the sentence and token levels from news articles, considering their fine granularity. Previous research has proposed various approaches ranging from traditional machine learning to deep learning, targeting event detection at these levels. Among these approaches, transformer-based approaches performed best, utilising transformers’ transferability and context awareness, and achieved state-of-the-art results. However, they considered sentence and token level tasks as separate tasks even though their interconnections can be utilised for mutual task improvements. To fill this gap, we propose a novel learning strategy named Two-phase Transfer Learning (TTL) based on transformers, which allows the model to utilise the knowledge from a task at a particular data granularity for another task at different data granularity, and evaluate its performance in sentence and token level event detection. Also, we empirically evaluate how the event detection performance can be improved for different languages (high- and low-resource), involving monolingual and multilingual pre-trained transformers and language-based learning strategies along with the proposed learning strategy. Our findings mainly indicate the effectiveness of multilingual models in low-resource language event detection. Also, TTL can further improve model performance, depending on the involved tasks’ learning order and their relatedness concerning final predictions.

show abstract

Multilingual Protest News Detection - Shared Task 1, CASE 2021

Hürriyetoğlu¹,

Mutlu²,

Yörük³

et al. 2021

Proceedings of the 4th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events From Text (CAS

View full text Add to dashboard Cite

Benchmarking state-of-the-art text classification and information extraction systems in multilingual, cross-lingual, few-shot, and zeroshot settings for socio-political event information collection is achieved in the scope of the shared task Socio-political and Crisis Events Detection at the workshop CASE @ ACL-IJCNLP 2021. Socio-political event data is utilized for national and international policyand decision-making. Therefore, the reliability and validity of such datasets are of utmost importance. We split the shared task into three parts to address the three aspects of data collection (Task 1), fine-grained semantic classification (Task 2), and evaluation (Task 3). Task 1, which is the focus of this report, is on multilingual protest news detection and comprises four subtasks that are document classification (subtask 1), sentence classification (subtask 2), event sentence coreference identification (subtask 3), and event extraction (subtask 4). All subtasks have English, Portuguese, and Spanish for both training and evaluation data. Data in Hindi language is available only for the evaluation of subtask 1. The majority of the submissions, which are 238 in total, are created using multi-and cross-lingual approaches. Best scores are between 77.27 and 84.55 F1-macro for subtask 1, between 85.32 and 88.61 F1macro for subtask 2, between 84.23 and 93.03 CoNLL 2012 average score for subtask 3, and between 66.20 and 78.11 F1-macro for subtask 4 in all evaluation settings. The performance of the best system for subtask 4 is above 66.20 F1 for all available languages. Although there is still a significant room for improvement in cross-lingual and zero-shot settings, the best submissions for each evaluation scenario yield remarkable results. Monolingual models outperformed the multilingual models in a few evaluation scenarios, in which there is relatively much training data.

show abstract

IIITT at CASE 2021 Task 1: Leveraging Pretrained Language Models for Multilingual Protest Detection

Cited by 3 publications

References 25 publications

Attentive fine-tuning of Transformers for Translation of low-resourced languages @LoResMT 2021

Attentive fine-tuning of Transformers for Translation of low-resourced languages @LoResMT 2021

TTL: transformer-based two-phase transfer learning for cross-lingual news event detection

Multilingual Protest News Detection - Shared Task 1, CASE 2021

Contact Info

Product

Resources

About