Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers

Abstract

In this article we present the design and the development of a knowledge based computational linguistic tool, Mlphon [em.el.foːɳ] for Malayalam language. Mlphon computationally models linguistic rules using finite state transducers and performs multiple functions including grapheme to phoneme (g2p) and phoneme to grapheme (p2g) conversions, syllabification, phonetic feature analysis and script grammar check. This open source software tool, released under MIT license, is developed as a one-stop solution to handle different speech related text processing tasks for automatic speech recognition, text to speech synthesis and nonspeech natural language processing tasks including syllable subword based language modeling, phoneme diversity analysis and text sanity check. The tool is evaluated on a manually crafted gold standard lexicon. Mlphon performs orthographic syllabification with 99% accuracy with a syllable error rate of 0.62% on the gold standard lexicon. For grapheme to phoneme conversion task, overall phoneme recognition accuracy of 99% with a phoneme error rate of 0.55% is obtained on gold standard lexicon. Additionally an extrinsic evaluation of Mlphon is performed by employing the pronunciation lexicon created using Mlphon, in Malayalam automatic speech recognition (ASR) task. Performance analysis in terms of the computation time of lexicon creation process and the word error rate (WER) on ASR task are presented along with a comparison over other automated tools for lexicon creation. Pronunciation lexicons with more than 100k commonly used Malayalam words in phonemised and syllabified forms is created and they are published as open language resources along with this work. We also demonstrate the usage of Mlphon on different natural language processing applications -syllable subword ASR, assisted pronunciation learning, phoneme diversity analysis and text sanity check. Being a knowledge based solution with open source code, Mlphon can be adapted to other languages of similar script nature.

WITHDRAWN: Integrating Prior Knowledge and Data-Driven Approaches for Improving Grapheme-to-Phoneme Conversion in Korean Language

“…Feng Wei et al [17] integrated a Russian word-sound translation technique using a weighted fi-nite-state transformer (WFST) with "many-to-many" alignment. Manohar K et al [18] developed 'Mlphon,' an open-source grapheme-to-phoneme conversion tool for Mal-ayalam. Additionally, Chinese scholars Li Peng et al [19] and Zhao Kun et al [20] ex-plored decision trees and random forests for English grapheme-to-phoneme conversion, while Wang Yongsheng et al [21] presented a supervised method, albeit without specific algorithm details.…”

Exploring corpus linguistics via computational tool analysis: key finding review

“…These tools can discern patterns and meanings in a manner like that of human comprehension [30]. Among the most notable NLP tools are Wordify [8], Mlphon [31], morphological analyzer [32], Runyakitara tool [33], LexTutor [34], [35], Coh-Metric [36], [37] linguistic queries and word count (LIWC) [38], UAM corpus tool [39], [40], SketchEngine (SkE) [41], Wmatrix [37] MultiAzterTest [42], sublanguage corpus analysis toolkit (SubCAT) [43], EnvText [44], InLang [45], Berri corpus manager [46], UCREL semantic analysis system (USAS) [47], PyMongo (Mongo DB, Python technology, Flask) [46], LancsBox 5.1.2 [48], LancsBox 4.5 [49], LancsBox [50], NooJ platform [51] and Bi-LSTM [52].…”

“…All tools, including concordance, NLP, and technology tools, were used by the researcher based on the articles found. Some studies have developed new corpus tools based on the current need to solve problems in a particular language such as Wordify [8], Mlphon [31], Runyakitara tool [33], PyMongo (Mongo DB, Python technology, Flask) [46], InLang [45], MultiAzterTest [42], PyLDAvis [58], and NooJ [51]. The types and functions of these computational tools are summarized in a mind map as shown in Figure 2.…”

Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023)

“…Similar efforts use an FST to develop a morphological generator and analyzer while simultaneously addressing the issue of missing diacritics (Alkhairy et al, 2020), demonstrating the easy expansion of an FST to create more resources for a language. Manohar et al (2022) extend the use of FSTs to text-to-speech (TTS) applications in low-resource settings, generating a model that converts between Malayalam phonemes and graphemes.…”