Arabic Dialect Identification Using iVectors and ASR Transcripts

Malmasi, Shervin; Zampieri, Marcos

doi:10.18653/v1/w17-1222

Cited by 32 publications

(22 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…System Description Paper ahaqst (Hanani et al, 2017) bayesline -CECL (Bestgen, 2017) cic ualg (Gómez-Adorno et al, 2017) Citius Ixa Imaxin (Gamallo et al, 2017) CLUZH (Clematide and Makarov, 2017) CUNI (Rosa et al, 2017) deepCybErNet -gauge -Helsinki-CLP (Tiedemann, 2017) MAZA (ADI) (Malmasi and Zampieri, 2017a) MAZA (GDI) (Malmasi and Zampieri, 2017b) mm lct (Medvedeva et al, 2017) qcri mit -SUKI (Jauhiainen et al, 2017) timeflow (Criscuolo and Aluisio, 2017) tubasfs Rama, 2017) unibuckernel (Ionescu andButnaru, 2017) XAC Bayesline (Barbaresi, 2017) Total 11 6 10 3 15 Table 1: The teams that participated in the VarDial'2017 Evaluation Campaign.…”

Section: Team Dsl Adi Gdi Clpmentioning

confidence: 99%

See 1 more Smart Citation

Findings of the VarDial Evaluation Campaign 2017

Zampieri¹,

Malmasi²,

Ljubešić³

et al. 2017

Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)

Self Cite

107

158

View full text Add to dashboard Cite

We present the results of the VarDial Evaluation Campaign on Natural Language Processing (NLP) for Similar Languages, Varieties and Dialects, which we organized as part of the fourth edition of the VarDial workshop at EACL'2017. This year, we included four shared tasks: Discriminating between Similar Languages (DSL), Arabic Dialect Identification (ADI), German Dialect Identification (GDI), and Cross-lingual Dependency Parsing (CLP). A total of 19 teams submitted runs across the four tasks, and 15 of them wrote system description papers.

show abstract

Section: Team Dsl Adi Gdi Clpmentioning

confidence: 99%

“…They used character 1-8-grams, word unigrams, and i-vectors. More detail about the system can be found in (Malmasi and Zampieri, 2017a). …”

Section: Participants and Approachesmentioning

confidence: 99%

Findings of the VarDial Evaluation Campaign 2017

Zampieri¹,

Malmasi²,

Ljubešić³

et al. 2017

Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)

Self Cite

107

158

View full text Add to dashboard Cite

show abstract

“…Comparing the results with known works used the same data set is [3] and [17].The results in [3] achieved a total accuracy of 60.2%. The results were achieved using sophisticated features extraction approach which involved human intervention, in addition of fusing the scores of a senone-based system and the SVM based i-vector system.…”

Section: Experiments Resultsmentioning

confidence: 82%

Spoken Arabic Dialect Identification Using Motif Discovery

Moftah

Fakhre²,

Elramly

2018

The Egyptian Journal of Language Engineering

View full text Add to dashboard Cite

In traditional Dialect Identification (DID) approaches, regardless of the level and type of features used for identification, they use either predefined references such as phones, phonemes, or even acoustic sounds that characterize a language/dialect, or involve some sort of transcription of the input data. The transcription may be manual or automatic using tools such as ASRs, Tokenizers, or Phone Recognizers. In this paper, we introduce a new approach based on analyzing the speech signal directly and extracting the features that characterize the dialect without any predefined references and without any sort of transcription. The main idea is that we find the repeated sequences (motifs) of the dialect by treating the speech signal as a times series, so we can apply motif discovery techniques to extract the repeated sequences directly from the speech signal. For motif extraction, we adopted an extremely fast parameter-free Self-Join motif discovery algorithm called Scalable Time series Ordered-search Matrix Profile (STOMP). We implemented the new approach in two stages; in the first we built a base line system in which we extracted 12 Mel Frequency Cepstral Coefficients (MFCC) from each motif, in the second stage we built an improved system using 39 coefficients by adding 13 Delta coefficients, 13 Delta-Delta coefficients, and 1 Log Energy coefficient. In both systems, we used Gaussian Mixture Model-Universal Background Model (GMM-UBM) as a classifier. We applied our new approach on three different motif lengths 500ms, 1000ms, and 1500ms using 1gmm component up to 2048gmm components. We downloaded the data set from Qatar-Computing-Research-Institute domain. We carried out our experiments on different Arabic dialects: the Egyptian (EGY), Gulf (GLF), Levantine (LEV), and North African (NOR).The base line results were very competitive with the traditional, more sophisticated approaches, while the improved system showed very good result. The improvement was so significant that we can consider the new approach as competitive, simple, and dialect-independent approach.

show abstract

“…Several studies, such as Malmasi, Refaee, and Dras (2015) and Malmasi and Zampieri (2016, 2017), have addressed Arabic dialect identification. Ali (2018) used a character-level convolutional neural network approach to classify Arabic dialects; this achieved an F 1-score of 57.6%.…”

Section: Related Workmentioning

confidence: 99%

Compression versus traditional machine learning classifiers to detect code-switching in varieties and dialects: Arabic as a case study

et al. 2020

View full text Add to dashboard Cite

The occurrence of code-switching in online communication, when a writer switches among multiple languages, presents a challenge for natural language processing tools, since they are designed for texts written in a single language. To answer the challenge, this paper presents detailed research on ways to detect code-switching in Arabic text automatically. We compare the prediction by partial matching (PPM) compression-based classifier, implemented in Tawa, and a traditional machine learning classifier sequential minimal optimization (SMO), implemented in Waikato Environment for Knowledge Analysis, working specifically on Arabic text taken from Facebook. Three experiments were conducted in order to: (1) detect code-switching among the Egyptian dialect and English; (2) detect code-switching among the Egyptian dialect, the Saudi dialect, and English; and (3) detect code-switching among the Egyptian dialect, the Saudi dialect, Modern Standard Arabic (MSA), and English. Our experiments showed that PPM achieved a higher accuracy rate than SMO with 99.8% versus 97.5% in the first experiment and 97.8% versus 80.7% in the second. In the third experiment, PPM achieved a lower accuracy rate than SMO with 53.2% versus 60.2%. Code-switching between Egyptian Arabic and English text is easiest to detect because Arabic and English are generally written in different character sets. It is more difficult to distinguish between Arabic dialects and MSA as these use the same character set, and most users of Arabic, especially Saudis and Egyptians, frequently mix MSA with their dialects. We also note that the MSA corpus used for training the MSA model may not represent MSA Facebook text well, being built from news websites. This paper also describes in detail the new Arabic corpora created for this research and our experiments.

show abstract

Arabic Dialect Identification Using iVectors and ASR Transcripts

Cited by 32 publications

References 27 publications

Findings of the VarDial Evaluation Campaign 2017

Findings of the VarDial Evaluation Campaign 2017

Spoken Arabic Dialect Identification Using Motif Discovery

Compression versus traditional machine learning classifiers to detect code-switching in varieties and dialects: Arabic as a case study

Contact Info

Product

Resources

About