Code Switch Point Detection in Arabic

Elfardy, Heba; Al-Badrashiny, Mohamed; Diab, Mona

doi:10.1007/978-3-642-38824-8_51

Cited by 23 publications

(21 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, Zbib et al (2012) show that small amounts of data from the right dialect can have a dramatic impact on the quality of Dialectal Arabic Machine Translation systems. Finally, we view the DSL task as a first step towards building a system that can identify code-switching in, for example, social media data, a task which has recently received increased attention from the NLP community 2 (Elfardy et al, 2013).…”

Section: Introductionmentioning

confidence: 99%

The NRC System for Discriminating Similar Languages

Goutte

Léger²,

Carpuat

2014

Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects

View full text Add to dashboard Cite

We describe the system built by the National Research Council Canada for the "Discriminating between similar languages" (DSL) shared task. Our system uses various statistical classifiers and makes predictions based on a two-stage process: we first predict the language group, then discriminate between languages or variants within the group. Language groups are predicted using a generative classifier with 99.99% accuracy on the five target groups. Within each group (except English), we use a voting combination of discriminative classifiers trained on a variety of feature spaces, achieving an average accuracy of 95.71%, with per-group accuracy between 90.95% and 100% depending on the group. This approach turns out to reach the best performance among all systems submitted to the open and closed tasks.

show abstract

Section: Introductionmentioning

confidence: 99%

The NRC System for Discriminating Similar Languages

Goutte

Léger²,

Carpuat

2014

Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects

View full text Add to dashboard Cite

show abstract

“…Previous work on Arabic dialect identification uses n-gram based features at both word-level and character-level to identify dialectal sentences (Elfardy et al, 2013;Cotterell et al, 2014;Zaidan et al, 2014). created a dataset of dialectal Arabic.…”

Section: Previous Workmentioning

confidence: 99%

“…They performed cross-validation experiments for dialect identification using word n-gram based features. Elfardy et al (2013) built a system to distinguish between Egyptian and MSA. They used word n-gram features combined with core (token-based and perplexitybased features) and meta features for training.…”

Section: Previous Workmentioning

confidence: 99%

“…For example, transcribing dialectal speech or automatically translating into a particular dialect would be aided by the use of targeted language models that are trained on texts in that dialect. This has led to recent interest in the automatic collection large dialectal corpora and the identification of different Arabic dialects (Al-Mannai et al, 2014;Elfardy et al, 2013;Cotterell et al, 2014;Zaidan et al, 2014).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP)

2014

View full text Add to dashboard Cite

There has been a lot of progress in the last 15 years in the area of Arabic Natural Language Processing (NLP). In particular, the TIDES, GALE, and BOLT programs provided a significant boost to Arabic NLP, both in generating new language and speech resources on a large scale, and in advancing the stateof-the-art in morphological processing, parsing, named entity recognition, information retrieval, speech recognition, and machine translation. The substantial investment done through these projects reflect the fact that the Middle East continues to grow in its political and economic importance. We also observe that countries in the Middle East invest substantially into higher education and into building an ecosystem, which fosters new research initiatives. This creates the hope that our own research field, NLP, and especially Arabic NLP will continue to grow, will continue to attract students both in the region and in top international universities, and that new job opportunities will open up not only in the well established language service providers, but also through start-ups offering innovative solutions.A number of Arabic NLP (or Arabic NLP-related) workshops and conferences have taken place, both in the Arab World and in association with international conferences. The Arabic NLP workshop at EMNLP 2014 follows in the footsteps of these previous efforts to provide a forum for researchers to share and discuss their ongoing work. The Arabic NLP workshop also includes a shared task on Automatic Arabic Error Correction, which was designed in the tradition of high profile NLP shared tasks such as CONLL's grammar/error detection and correction shared tasks in 2011-2013, and numerous machine translation campaigns by NIST/WMT/MEDAR, among others. The challenge chosen for the shared task is highly relevant, not only to spelling correction while composing a text, but also to developing techniques for automatically correcting errors in the far-from-perfect outputs of NLP technologies such as speech recognition or machine translation.We are happy to have received 40 submissions. Unfortunately, not all the papers could be included in the workshop due to time limitations. The acceptance rate is 50%. The papers cover a wide range of topics: building language resources for standard and dialectal Arabic, language identification, sentiment analysis, named entity disambiguation, and machine translation for dialectal Arabic, etc. Twelve papers were selected for oral presentation and were organized under the general topics Corpora (four papers), Text Mining (four papers), Translation & Transliteration (three papers) and one paper describing the shared task. The remaining eight papers were selected to be presented in a poster session. There is no difference in quality between the oral and poster presentations.The shared task was a success. We received 18 systems submissions from nine teams in six countries, representing a diverse set of approaches. Nine shared task system description (short) papers are included in the proceedings to ...

show abstract

“…Several studies has emerged on word-level language identification (Nguyen and Dogruöz, 2013;Das and Gambäck, 2014;cf. Solorio et al, 2014), predicting codeswitching points (Solorio and Liu, 2008a;Elfardy et al, 2013), and POS tagging (Solorio and Liu, 2008b;Vyas et al, 2014;Jamatia et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

Part of Speech Annotation of a Turkish-German Code-Switching Corpus

Çetinoğlu

Çöltekin

2016

Proceedings of the 10th Linguistic Annotation Workshop Held In Conjunction With ACL 2016 (LAW-X 2016)

View full text Add to dashboard Cite

In this paper we describe our efforts on POS annotation of a code-switching corpus created from Turkish-German tweets. We use Universal Dependencies (UD) POS tags as our tag set. While the German parts of the corpus employ UD specifications, for the Turkish parts we propose annotation guidelines that adopt UD's language-general rules when it is applicable and adapt its principles to Turkishspecific phenomena when it is not. The resulting corpus has POS annotation of 1029 tweets, which is aligned with existing language identification annotation.

show abstract

Code Switch Point Detection in Arabic

Cited by 23 publications

References 1 publication

The NRC System for Discriminating Similar Languages

The NRC System for Discriminating Similar Languages

Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP)

Part of Speech Annotation of a Turkish-German Code-Switching Corpus

Contact Info

Product

Resources

About