A New Multi Varied Arabic Corpus

Meskaldji, Khouloud; Chikhi, Salim; Bensalem, Imene

doi:10.1109/pais.2018.8598524

Cited by 3 publications

(2 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When scraping Arabic sites, text encodings must be in UTF-8 for the text to be processed by NLP. This also accounts for the Arabic text direction, from right to left, and proper encoding ensures that this feature is recognized (Meskaldji et al, 2018). Several technical issues are that, 1) Arabic sites store limited data due to high database costs; 2) Security features on many Arabic sites can hinder scraping efforts.…”

Section: Corpora Buildingmentioning

confidence: 99%

ConfliBERT-Arabic: A Pre-trained Arabic Language Model for Politics, Conflicts and Violence

Alsarra,

Abdeljaber,

Yang

et al. 2023

Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Proce

View full text Add to dashboard Cite

This study investigates the use of Natural Language Processing (NLP) methods to analyze politics, conflicts and violence in the Middle East using domain-specific pre-trained language models. We introduce Arabic text and present ConfliBERT-Arabic, a pre-trained language models that can efficiently analyze political, conflict and violence-related texts. Our technique hones a pre-trained model using a corpus of Arabic texts about regional politics and conflicts. Performance of our models is compared to baseline BERT models. Our findings show that the performance of NLP models for Middle Eastern politics and conflict analysis are enhanced by the use of domain-specific pre-trained local language models. This study offers political and conflict analysts, including policymakers, scholars, and practitioners new approaches and tools for deciphering the intricate dynamics of local politics and conflicts directly in Arabic.

show abstract

Section: Corpora Buildingmentioning

confidence: 99%

ConfliBERT-Arabic: A Pre-trained Arabic Language Model for Politics, Conflicts and Violence

Alsarra,

Abdeljaber,

Yang

et al. 2023

Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Proce

View full text Add to dashboard Cite

show abstract

“…This corpus can be used for the text classification process. In addition, Arabic Text Corpus [35] is an Arabic text corpus with more than 233k words built from three different sources: Quranic text, Classical Arabic text, and Modern Arabic text. The corpus was collected from the Quran, contemporary Arabic corpora, and the InAra Arabic corpus.…”

Section: Literature Reviewmentioning

confidence: 99%

An Incremental Approach to Corpus Design and Construction: Application to a Large Contemporary Saudi Corpus

et al. 2021

View full text Add to dashboard Cite

Due to the rapid developments in technology and the sudden expansion of social media use, Dialect Arabic has become an important source of data that needs to be addressed when building Arabic corpora. In this paper, thirty-three Arabic corpora are surveyed to show that despite all of the developments in the literature, Saudi dialect (SD) corpora still need further expansion. This paper contributes to the literature on SD corpora by creating the largest Saudi corpus -the King Saud University Saudi Corpus (KSUSC)with +1B total words, including +119M SD words. The KSUSC not only is the newest and largest SD corpus but is also diverse, covering 26 domains in text collected from five different sources. This paper also contributes to the literature by developing a new incremental preprocessing system that is used to create relevant lexicons that are then used to clean and normalize the collected data. This incremental system is scalable and can be adapted for different resources and dialects. Moreover, the collection process for building the KSUSC is discussed in detail, and the challenges in collecting SD text with respect to each platform are highlighted. By the end of this paper, different design criteria are proposed and used with the KSUSC to conclude that the resulting corpus can be of great benefit to researchers who are interested in integrating the corpus with their own work or using its resulting lexicons with Saudi-based NLP tasks.

show abstract

Corpus Construction and Annotation Challenge for Language Identification and Sentiment Analysis

Touahri

Mazroui

2020

Advances in Smart Technologies Applications and Case Studies

View full text Add to dashboard Cite

A New Multi Varied Arabic Corpus

Cited by 3 publications

References 8 publications

ConfliBERT-Arabic: A Pre-trained Arabic Language Model for Politics, Conflicts and Violence

ConfliBERT-Arabic: A Pre-trained Arabic Language Model for Politics, Conflicts and Violence

An Incremental Approach to Corpus Design and Construction: Application to a Large Contemporary Saudi Corpus

Corpus Construction and Annotation Challenge for Language Identification and Sentiment Analysis

Contact Info

Product

Resources

About