Unsupervised discovery of Persian morphemes

Arabsorkhi, Mohsen; Shamsfard, Mehrnoush

doi:10.3115/1608974.1609002

Cited by 3 publications

(2 citation statements)

References 11 publications

(6 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Finite-State Morphology (FSM) has covered both morphological analysis and generation aspects of computational morphology addressing various languages some of which have been well-equipped by NLP and CL tools and some not. While resourceful languages such English [Beesley and Karttunen 2003;Minnen et al 2001] and German [Schmid 2005] are front-runners in FSM studies, we also observe similar scholarly attempts regarding other languages which might not be considered as widely-studied as English or German such as Uralic languages [Novák 2015], Arabic [Soudi et al 2007] and Persian [Arabsorkhi and Shamsfard 2006;Megerdoomian et al 2000]. This is also correct for less-studied languages such as Croatian [Mihajlović 2014].…”

Section: Related Worksupporting

confidence: 79%

Towards Finite-State Morphology of Kurdish

Ahmadi,

Hassani

2020

Preprint

View full text Add to dashboard Cite

Morphological analysis is the study of the formation and structure of words. It plays a crucial role in various tasks in Natural Language Processing (NLP) and Computational Linguistics (CL) such as machine translation and text and speech generation. Kurdish is a less-resourced multi-dialect Indo-European language with highly inflectional morphology. In this paper, as the first attempt of its kind, the morphology of the Kurdish language (Sorani dialect) is described from a computational point of view. We extract morphological rules which are transformed into finite state transducers for generating and analyzing words. The result of this research assists in conducting studies on language generation for Kurdish and enhances the Information Retrieval (IR) capacity for the language while leveraging the Kurdish NLP and CL into a more advanced computational level.

show abstract

Section: Related Worksupporting

confidence: 79%

Towards Finite-State Morphology of Kurdish

Ahmadi,

Hassani

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Although they segmented very low portion of Persian words (only some Persian verbs), the quality of their machine translation system increases by 1.9 points of BLEU score. Arabsorkhi and Shamsfard (2006) proposed a Minimum Description Length (MDL) based algorithm with some improvements for discovering the morphemes of Persian language through automatic analysis of corpora.…”

Section: Related Workmentioning

confidence: 99%

Supervised Morphological Segmentation Using Rich Annotated Lexicon

Ansari

Žabokrtský²,

Mahmoudi

et al. 2019

Proceedings - Natural Language Processing in a Deep Learning World

View full text Add to dashboard Cite

Morphological segmentation of words is the process of dividing a word into smaller units called morphemes; it is tricky especially when a morphologically rich or polysynthetic language is under question. In this work, we designed and evaluated several Recurrent Neural Network (RNN) based models as well as various other machine learning based approaches for the morphological segmentation task. We trained our models using annotated segmentation lexicons. To evaluate the effect of the training data size on our models, we decided to create a large hand-annotated morphologically segmented corpus of Persian words, which is, to the best of our knowledge, the first and the only segmentation lexicon for the Persian language. In the experimental phase, using the hand-annotated Persian lexicon and two smaller similar lexicons for Czech and Finnish languages, we evaluated the effect of the training data size, different hyperparameters settings as well as different RNN-based models.

show abstract