CALYOU: A Comparable Spoken Algerian Corpus Harvested from YouTube

Abidi, Karima; Menacer, Mohamed Amine; Smaïli, Kamel

doi:10.21437/interspeech.2017-1305

Cited by 14 publications

(18 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The resulting corpus contains 5.6 million sentences Suwaileh et al (Suwaileh et al, 2016) Saad and Alijla (2017) propose the construction of a comparable Wikipedia corpus between the Arabic and Egyptian dialects. This corpus contains 10,197 aligned documents Another comparable corpus (CALYOU) was constructed by Abidi et al (2017). This corpus is dedicated to spoken Algerian To align messages, the authors used different approaches such as: dictionary based, indexing words by their sounds and finally an approach based on the similarity proposed by word2vec (Mikolov et al, 2013).…”

Section: Building Resourcesmentioning

confidence: 99%

“…In the context of MT, Meftouh et al (2018) used a phrasebased MT system, GIZA++ (Och and Ney, 2003) for alignment and SRILM toolkit (Stolcke, 2002) The best results that these authors obtained were between the Algiers dialect and the dialect of Annaba (with BLEU score up to 67.31) which is perfectly understandable where both dialects are spoken into the same country (Algeria). To the best of our knowledge, Saad and Alijla (2017), Abidi et al (2017), Bouamor et al (2018), Kumar et al (2014) have not proposed any system to validate their comparable and parallel corpora.…”

Section: Semantic-level Analysismentioning

confidence: 99%

See 1 more Smart Citation

Arabic natural language processing: An overview

Guellil

Saadane²,

Azouaou

et al. 2021

Journal of King Saud University - Computer and Information Scie

View full text Add to dashboard Cite

a b s t r a c tArabic is recognised as the 4th most used language of the Internet. Arabic has three main varieties: (1) classical Arabic (CA), (2) Modern Standard Arabic (MSA), (3) Arabic Dialect (AD). MSA and AD could be written either in Arabic or in Roman script (Arabizi), which corresponds to Arabic written with Latin letters, numerals and punctuation. Due to the complexity of this language and the number of corresponding challenges for NLP, many surveys have been conducted, in order to synthesise the work done on Arabic. However these surveys principally focus on two varieties of Arabic (MSA and AD, written in Arabic letters only), they are slightly old (no such survey since 2015) and therefore do not cover recent resources and tools. To bridge the gap, we propose a survey focusing on 90 recent research papers (74% of which were published after 2015). Our study presents and classifies the work done on the three varieties of Arabic, by concentrating on both Arabic and Arabizi, and associates each work to its publicly available resources whenever available.

show abstract

Section: Building Resourcesmentioning

confidence: 99%

Section: Semantic-level Analysismentioning

confidence: 99%

Arabic natural language processing: An overview

Guellil

Saadane²,

Azouaou

et al. 2021

Journal of King Saud University - Computer and Information Scie

View full text Add to dashboard Cite

show abstract

“…-The comparable corpus CALYOU CALYOU 3 [1] is an Algerian dialect comparable corpus of Youtube comments. It was collected by querying Youtube with key-words related to current Algerian events.…”

Section: Data Descriptionmentioning

confidence: 99%

“…They are written in Arabic and Latin script. 1 Also, They are written sometimes with a mixture of letters and numbers. Arab people exploit the similarity between some Arabic letters and numbers to write the dialect, for example similarity between 3 and , 7 and and 9 and .…”

Section: Introductionmentioning

confidence: 99%

Script Independent Morphological Segmentation for Arabic Maghrebi Dialects: An Application to Machine Translation

Harrat¹,

Meftouh²,

Smaïli³

2019

CyS

View full text Add to dashboard Cite

This research deals with resources creation for under-resourced languages. We try to adapt existing resources for other resourced-languages to process less-resourced ones. We focus on Arabic dialects of the Maghreb, namely Algerian, Moroccan and Tunisian. We first adapt a well-known statistical word segmenter to segment Algerian dialect texts written in both Arabic and Latin scripts. We demonstrate that unsupervised morphological segmentation could be applied to Arabic dialects regardless of used script. Next, we use this kind of segmentation to improve statistical machine translation scores between the tree Maghrebi dialects and French. We use a parallel multidialectal corpus that includes six Arabic dialects in addition to MSA and French. We achieved interesting results. Regards to word segmentation, the rate of correctly segmented words reached 70% for those written in Latin script and 79% for those written in Arabic script. For machine translation, the unsupervised morphological segmentation helped to decrease out-of-vocabulary words rates by a minimum of 35%.

show abstract

“…That is why, our first objective is to provide a code-switched dataset. Code-switched corpora are either generated in an artificial way [13,12] or collected from social media and/or online texts [1]. Once the data are collected, the processing of code-switching is carried out by adapting the existing models and tools or by proposing new ones.…”

Section: Introductionmentioning

confidence: 99%

Machine Translation on a Parallel Code-Switched Corpus

Menacer

Langlois

Jouvet

et al. 2019

Advances in Artificial Intelligence

View full text Add to dashboard Cite

Code-switching (CS) is the phenomenon that occurs when a speaker alternates between two or more languages within an utterance or discourse. In this work, we investigate the existence of code-switching in formal text, namely proceedings of multilingual institutions. Our study is carried out on the Arabic-English code-mixing in a parallel corpus extracted from official documents of United Nations. We build a parallel code-switched corpus with two reference translations one in pure Arabic and the other in pure English. We also carry out a human evaluation of this resource in the aim to use it to evaluate the translation of code-switched documents. To the best of our knowledge, this kind of corpora does not exist. The one we propose is unique. This paper examines several methods to translate codeswitched corpus: conventional statistical machine translation, the end-to-end neural machine translation and multitask-learning.

show abstract

CALYOU: A Comparable Spoken Algerian Corpus Harvested from YouTube

Cited by 14 publications

References 8 publications

Arabic natural language processing: An overview

Arabic natural language processing: An overview

Script Independent Morphological Segmentation for Arabic Maghrebi Dialects: An Application to Machine Translation

Machine Translation on a Parallel Code-Switched Corpus

Contact Info

Product

Resources

About