An Exploratory Approach to Find a Novel Metric Based Optimum Language Model for Automatic Bangla Word Prediction

Abstract

Abstract-Word completion and word prediction are two important phenomena in typing that have intense effect on aiding disable people and students while using keyboard or other similar devices. Such auto completion technique also helps students significantly during learning process through constructing proper keywords during web searching. A lot of works are conducted for English language, but for Bangla, it is still very inadequate as well as the metrics used for performance computation is not rigorous yet. Bangla is one of the mostly spoken languages (3.05% of world population) and ranked as seventh among all the languages in the world. In this paper, word prediction on Bangla sentence by using stochastic, i.e. N-gram based language models are proposed for auto completing a sentence by predicting a set of words rather than a single word, which was done in previous work. A novel approach is proposed in order to find the optimum language model based on performance metric. In addition, for finding out better performance, a large Bangla corpus of different word types is used.

An investigative design of optimum stochastic language model for bangla autocomplete

“…Therefore, accuracy and failure rate are used in order to address this issue. If is mention that the equation 2 & 3 used from our previous paper [3]. If w m matches with v mi , (i.e.…”

“…This is why the average word length of corpus is used in looping condition. In this way, n*, the considerable optimum value of n is automatically calculated for every language model stated earlier, which is given as pseudocode in Algorithm 1 of [3]. The best model is chosen by the technique, which was describe in our previous paper [3].…”

“…Probability is based on counting things or word in most cases. In our previous works [2,3], we used different types of language models for word prediction. Both these two works are different from the work presented in this paper because word prediction obviously differs from autocomplete, i.e.…”

“…word completion. In these earlier works word [2] or word set [3] is predicted based on one or more preceding words, but in this work, word fragment set is being predicted based on one or more words and a single character.…”

“…13, No. 2, February 2019 : 671 -676 674 test set, through which the considerable optimum prediction length (n*) is determined automatically based on Algorithm 1of[3].The entire approach is shown in Figure2.…”