Improving Arabic Lemmatization Through a Lemmas Database and a Machine-Learning Technique

Namly, Driss; Bouzoubaa, Karim; Jihad, Abdelhamid El; Aouragh, Si Lhoussain

doi:10.1007/978-3-030-34614-0_5

Cited by 9 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In comparison, lemmatization takes O(MN) time because the algorithm reads in the whole sentence to identify the context and POS tag beforehand. A compromise is context-free lemmatization, which produces the appropriate lemmas without using the whole context (Namly et al 2020). By leveraging existing inflection tables, this method has achieved some success in English, Dutch, and German (Nicolai and Kondrak 2016).…”

Section: Comparison Of Stemming and Lemmatizationmentioning

confidence: 99%

Comparison of text preprocessing methods

Chai

2022

Nat. Lang. Eng.

View full text Add to dashboard Cite

Text preprocessing is not only an essential step to prepare the corpus for modeling but also a key area that directly affects the natural language processing (NLP) application results. For instance, precise tokenization increases the accuracy of part-of-speech (POS) tagging, and retaining multiword expressions improves reasoning and machine translation. The text corpus needs to be appropriately preprocessed before it is ready to serve as the input to computer models. The preprocessing requirements depend on both the nature of the corpus and the NLP application itself, that is, what researchers would like to achieve from analyzing the data. Conventional text preprocessing practices generally suffice, but there exist situations where the text preprocessing needs to be customized for better analysis results. Hence, we discuss the pros and cons of several common text preprocessing methods: removing formatting, tokenization, text normalization, handling punctuation, removing stopwords, stemming and lemmatization, n-gramming, and identifying multiword expressions. Then, we provide examples of text datasets which require special preprocessing and how previous researchers handled the challenge. We expect this article to be a starting guideline on how to select and fine-tune text preprocessing methods.

show abstract

Section: Comparison Of Stemming and Lemmatizationmentioning

confidence: 99%

Comparison of text preprocessing methods

Chai

2022

Nat. Lang. Eng.

View full text Add to dashboard Cite

show abstract

“…Arabic is an essential language as it is the native language of Arabic countries, with around 414 million people [28]. There are three primary forms of Arabic: Classical Arabic (CA), Modern Standard Arabic (MSA), and informal Arabic or dialects.…”

Section: ) Arabic Natural Language Processingmentioning

confidence: 99%

Arabic Rumor Detection Using Contextual Deep Bidirectional Language Modeling

et al. 2022

View full text Add to dashboard Cite

In today's world, news outlets have changed dramatically; newspapers are obsolete, and radio is no longer in the picture. People look for news online and on social media, such as Twitter and Facebook. Social media contributors share information and trending stories before verifying their truthfulness, thus, spreading rumors. Early identification of rumors from social media has attracted many researchers. However, a relatively smaller number of studies focused on other languages, such as Arabic. In this study, an Arabic rumor detection model is proposed. The model was built using transformer-based deep learning architecture. According to the literature, transformers are neural networks with outstanding performance in natural language processing tasks. Two transformers-based models, AraBERT and MARBERT, were employed, tested, and evaluated using three recently developed Arabic datasets. These models are extensions to the BERT, Bidirectional Encoder Representations from Transformers, a deep learning model that uses transformer architecture to learn the text representations and leverages the attention mechanism. We have also mitigated the challenges introduced by the imbalanced training datasets by employing two sampling techniques. The experimental results of our proposed approaches achieved 0.97 accuracy. This result demonstrated the effectiveness of the proposed method and outperformed other existing Arabic rumor detection methods.

show abstract

“…These stemmers are ARLSTem v1.0 [16], Tashaphyne, integrated system of rice intensification (ISRI) stemmer [17], and the stemmer included in Madamira [18]. c) Lemmatization: Lemmatization has recently proved to be beneficial for Arabic text classifiers [19]- [21].…”

Section: Text Processing Techniquesmentioning

confidence: 99%

Arabic authorship attribution on Twitter: what is really matters?

Kah¹,

airej

Zeroual

2022

IJEECS

View full text Add to dashboard Cite

Recently, authorship attribution (AA) of online social networks texts has gained more attention. However, since 2015, when the first work that addressed the AA of Arabic tweets was published, we found that nothing much has been done after that. Thus, the current paper presents an extensive study that investigates the effects of various factors on the AA of Arabic short-texts, especially tweets. This led to a proposed architecture in which the AA accuracy is examined depending on the size of the training dataset, the number of classes covered, the text processing techniques applied, the methods used for both feature selection and extraction, and finally, the classifier implemented. As a result, we performed 792 different tests. The highest accuracy recorded is 97.4%, and it is among the best results published so far.

show abstract

Improving Arabic Lemmatization Through a Lemmas Database and a Machine-Learning Technique

Cited by 9 publications

References 13 publications

Comparison of text preprocessing methods

Comparison of text preprocessing methods

Arabic Rumor Detection Using Contextual Deep Bidirectional Language Modeling

Arabic authorship attribution on Twitter: what is really matters?

Contact Info

Product

Resources

About