Building Statistical Language Models for Persian Continuous Speech Recognition Systems Using the Peykare Corpus

Bahrani, Mohammad; Sameti, Hossein

doi:10.1142/s1793840611002188

Cited by 4 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first edition of this corpus consists of about ten million words and it was increased to about 100 million words in the second [18,19] that are reduced to 166 POS tags in this work.…”

Section: Text Corpusmentioning

confidence: 99%

“…For example, the consonant gemination marker in the word "fann/vari" (technology) is removed resulting in the word "fan/vari" [19]. The multiplicity of the POS tags in the corpus was the next problem to be solved.…”

Section: Statistical Language Modelingmentioning

confidence: 99%

“…These modifications reduced the size of the tag set to 166. Finally, the following statistics were extracted from the corpus to build the LMs [18,19]: unigram statistics of words (The 20,000 most frequent words in the corpus were chosen as the vocabulary set); bigram statistics of words; trigram statistics of words; unigram statistics of POS tags (for 166 tags); bigram statistics of POS tags; trigram statistics of POS tags; number of assigning one POS tag to each word in the corpus (lexical generation statistics). After extracting the word-based n-gram statistics, the back-o trigram language model was built using Katz smoothing method [45].…”

Section: Statistical Language Modelingmentioning

confidence: 99%

“…In addition, the language model score for class-based bigram and trigram language models can be computed [19]. As shown in Figure 1 by dotted line, the statistical LM can be applied to the system at the end of the search by n-best re-scorer.…”

Section: Statistical Language Modelingmentioning

confidence: 99%

“…Various data compensation and model compensation methods are used to achieve this objective. Also classbased n-gram language models (LM) [18,19] with generalized phrase structure grammar (GPSG)-based Persian grammar [20] are utilized as word-level and sentencelevel linguistic information. The frameworks for testing and comparing the effects of the implemented methods and also for optimizing the parameters were gradually built up.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A large vocabulary continuous speech recognition system for Persian language

Sameti

Veisi

Bahrani

et al. 2011

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

The first large vocabulary speech recognition system for the Persian language is introduced in this paper. This continuous speech recognition system uses most standard and state-of-the-art speech and language modeling techniques. The development of the system, called Nevisa, has been started in 2003 with a dominant academic theme. This engine incorporates customized established components of traditional continuous speech recognizers and its parameters have been optimized for real applications of the Persian language. For this purpose, we had to identify the computational challenges of the Persian language, especially for text processing and extract statistical and grammatical language models for the Persian language. To achieve this, we had to either generate the necessary speech and text corpora or modify the available primitive corpora available for the Persian language. In the proposed system, acoustic modeling is based on hidden Markov models, and optimized decoding, pruning and language modeling techniques were used in the system. Both statistical and grammatical language models were incorporated in the system. MFCC representation with some modifications was used as the speech signal feature. In addition, a VAD was designed and implemented based on signal energy and zero-crossing rate. Nevisa is equipped with out-of-vocabulary capability for applications with medium or small vocabulary sizes. Powerful robustness techniques were also utilized in the system. Model-based approaches like PMC, MLLR and MAP, along with feature robustness methods such as CMS, PCA, RCC and VTLN, and speech enhancement methods like spectral subtraction and Wiener filtering, along with their modified versions, were diligently implemented and evaluated in the system. A new robustness method called PC-PMC was also proposed and incorporated in the system. To evaluate the performance and optimize the parameters of the system in noisy-environment tasks, four real noisy speech data sets were generated. The final performance of Nevisa in noisy environments is similar to the clean conditions, thanks to the various robustness methods implemented in the system. Overall recognition performance of the system in clean and noisy conditions assures us that the system is a real-world product as well as a competitive ASR engine.

show abstract

“…The first edition of this corpus consists of about ten million words and it was increased to about 100 million words in the second [18,19] that are reduced to 166 POS tags in this work.…”

Section: Text Corpusmentioning

confidence: 99%

Section: Statistical Language Modelingmentioning

confidence: 99%