An exploratory research on grammar checking of Bangla sentences using statistical language models

Abstract

N-gram based language models are very popular and extensively used statistical methods for solving various natural language processing problems including grammar checking. Smoothing is one of the most effective techniques used in building a language model to deal with data sparsity problem. Kneser-Ney is one of the most prominently used and successful smoothing technique for language modelling. In our previous work, we presented a Witten-Bell smoothing based language modelling technique for checking grammatical correctness of Bangla sentences which showed promising results outperforming previous methods. In this work, we proposed an improved method using Kneser-Ney smoothing based n-gram language model for grammar checking and performed a comparative performance analysis between Kneser-Ney and Witten-Bell smoothing techniques for the same purpose. We also provided an improved technique for calculating the optimum threshold which further enhanced the the results. Our experimental results show that, Kneser-Ney outperforms Witten-Bell as a smoothing technique when used with n-gram LMs for checking grammatical correctness of Bangla sentences.

Encapsulation of semantic description with syntactic components for the Arabic language

“…Traversing a tree-expression is called a sequence-expression (or merely an expression), traversing a tree-clause is called sentence-sequence (just a [18]). If in the grammar Snt (Λ μ ): a) The set L is the lexicon of the Arabic language, (3) -functions from the morphology of the Arabic language; b) Elements Xi * Yi (i = 1, … , k) are word combinations (pairs of related word forms) admissible in the syntax of the Arabic language; c) Linked pairs of word forms in Arabic expressions form a rooted tree, Then Snt (Λ μ ) should have a lot in common with the syntax of the Arabic language; therefore, Snt (Λ μ ) will be called the model of grammar of the Arabic language [7], [26]. It is not difficult to show the existence of (But in the grammar Snt (Λ μ ) there will be sentences of dubious meaning.).…”

Adaptive Language Processing Unit for Malaysian Sign Language Synthesizer

“…Variable-length patterns can be directives to specific rules, like regular expressions. However, they can also be random and depend on the context and pattern repetition in the patterns dictionary [45]- [47].…”

“…In computational linguistics, a sequence of 𝑛 contiguous words is called N-gram [46][47][48]. The system implements a combination of n-grams and NLP.…”