Lessons from building a Persian written corpus: Peykare

Bijankhan, M.; Sheykhzadegan, Javad; Bahrani, Mohammad; Ghayoomi, Masood

doi:10.1007/s10579-010-9132-x

Cited by 60 publications

(25 citation statements)

References 17 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this research several experiments were conducted to convert grapheme sequence to the phoneme sequence for the Persian language using CURRENNT. To train the model two corpuses are used: FarsDat [24] with more than 500,000 words; and labeled section of the Persian written corpus [25] which has over 10 million labeled words. The goal is to map a sequence of graphemes to a sequence of phonemes.…”

Section: Letter Pronunciation Candidatesmentioning

confidence: 99%

Persian sentences to phoneme sequences conversion based on recurrent neural networks

Behbahani

BabaAli

Turdalyuly

2016

Open Computer Science

View full text Add to dashboard Cite

Grapheme to phoneme conversion is one of the main subsystems of Text-to-Speech (TTS) systems. Converting sequence of written words to their corresponding phoneme sequences for the Persian language is more challenging than other languages; because in the standard orthography of this language the short vowels are omitted and the pronunciation of words depends on their positions in a sentence. Common approaches used in the Persian commercial TTS systems have several modules and complicated models for natural language processing and homograph disambiguation that make the implementation harder as well as reducing the overall precision of system. In this paper we define the grapheme-to-phoneme conversion as a sequential labeling problem; and use the modified Recurrent Neural Networks (RNN) to create a smart and integrated model for this purpose. The recurrent networks are modified to be bidirectional and equipped with Long-Short Term Memory (LSTM) blocks to acquire most of the past and future contextual information for decision making. The experiments conducted in this paper show that in addition to having a unified structure the bidirectional RNN-LSTM has a good performance in recognizing the pronunciation of the Persian sentences with the precision more than 98 percent.

show abstract

Section: Letter Pronunciation Candidatesmentioning

confidence: 99%

Persian sentences to phoneme sequences conversion based on recurrent neural networks

Behbahani

BabaAli

Turdalyuly

2016

Open Computer Science

View full text Add to dashboard Cite

show abstract

“…UPC is another Farsi corpus developed by Seraji et al [2012] and is a modified version of Bijankhan with additional sentences including 2.7M words annotated with 32 POS tags. 10 Another corpus that includes 10M tagged words is the Peykareh corpus [Bijankhan et al 2011], which unfortunately is not publicly available. In Farsi NLP tasks, the Bijankhan corpus is frequently used, and to make our work comparable, we also used this dataset.…”

Section: Training Corpusmentioning

confidence: 99%

Boosting Neural POS Tagger for Farsi Using Morphological Information

Passban

Way

2016

ACM Trans. Asian Low-Resour. Lang. Inf. Process.

View full text Add to dashboard Cite

is a low-resource language that suffers from the data sparsity problem and a lack of efficient processing tools. Due to their broad application in natural language processing tasks, part-of-speech (POS) taggers are one of those important tools that should be considered in this respect. Despite recent work on Farsi tagging, there is still room for improvement. The best reported accuracy so far is 96%, which in special cases can rise to 96.9%. The main problem with existing taggers is their inefficiency in coping with outof-vocabulary (OOV) words. Addressing both problems of accuracy and OOV words, we developed a neural network-based POS tagger (NPT) that performs efficiently on Farsi. Despite using less data, NPT provides better results in comparison to state-of-the-art systems. Our proposed tagger performs with an accuracy of 97.4%, with performance highly influenced by morphological features. We carry out a shallow morphological analysis and show considerable improvement over the baseline configuration.

show abstract

“…On the other hand, the lack of the tagging especially in half-space rule leaves more unedited multi-part words in evaluation step. Moreover, available Persian tagged corpus such as Peykare [5] does not comply with half-space character. In this paper, we propose a different statistical approach which uses a fertility-based IBM Model [6] as word alignment by employing a parallel corpus which is created for the special purpose of Persian multi-part word edition.…”

Section: ‫حاصل‬ ‫ضرب‬mentioning

confidence: 99%

A new model for persian multi-part words edition based on statistical machine translation

Zahedi

Arjomandzadeh

2016

JAIDM

View full text Add to dashboard Cite

Multi-part words in English language are hyphenated and hyphen is used to separate different parts. Persian language consists of multi-part words as well. Based on Persian morphology, half-space character is needed to separate parts of multi-part words where in many cases people incorrectly use space character instead of half-space character. This common incorrectly use of space leads to some serious issues in Persian text processing and text readability. In order to cope with the issues, this work proposes a new model to correct spacing in multi-part words. The proposed method is based on statistical machine translation paradigm. In machine translation paradigm, text in source language is translated into a text in destination language on the basis of statistical models whose parameters are derived from the analysis of bilingual text corpora. The proposed method uses statistical machine translation techniques considering unedited multi-part words as a source language and the space-edited multi-part words as a destination language. The results show that the proposed method can edit and improve spacing correction process of Persian multi-part words with a statistically significant accuracy rate.

show abstract

Lessons from building a Persian written corpus: Peykare

Cited by 60 publications

References 17 publications

Persian sentences to phoneme sequences conversion based on recurrent neural networks

Persian sentences to phoneme sequences conversion based on recurrent neural networks

Boosting Neural POS Tagger for Farsi Using Morphological Information

A new model for persian multi-part words edition based on statistical machine translation

Contact Info

Product

Resources

About