State-of-the-art English to Persian Statistical Machine Translation system

Abstract

Comparison of several kinds of English-PersianStatistical Machine Translation systems is reported in this paper. A large parallel corpus containing about 6 million tokens on each side has been developed for training the proposed SMT system. In development of the parallel corpus, a noisy filtering system based on MaxEnt classifier bas been innovated to distinguish between correct and incorrect sentence pairs. By using the generated parallel corpus, a variety of SMT systems on English to Persian languages has been developed. Several variations on SMT, such as hybrid MT or statistical post editing MT has been proposed in this paper. The whole systems were tested on two different types of test set, one extracted randomly from parallel corpus and the other containing formal English sentences extracted from English learning book. The results shows hybrid system of SMT augmented by a rule based detection of English phrasal verb and Persian compound verb improves the baseline significantly. Also, state-of-the-art results on English-Persian translation are obtained by Verb-aware SMT with respect to BLEU measure.

A learning to rank approach for cross-language information retrieval exploiting multiple translation resources

“…Table 2 shows the statistics of Hamshahri corpus. The 20M parallel corpus is constructed from four different parallel corpora: Roman parallel corpus (Mansouri and Faili 2012), Iran Telecommunication Research Center parallel corpus (Jabbari et al 2012), European Language Resources Association English–Persian parallel corpus (Mosavi Miangah 2009), and a part of Mizan parallel corpus b . This corpus consists of 1,109,584 aligned sentences, which have about 20,000,000 words on each side.…”