Named-Entity Recognition in Bengali

Abstract

This paper 1 describes two systems for Named Entity Recognition (NER) and performance of two systems has been compared. The first system is a rule-based one whereas the second one is statistical (based on CRF) in nature. The systems vary in some other aspects too, for example, the first system works on untagged data (not even POS tag is done) to identify NER whereas the second system makes use of a POS tagger and a chunker. The rules used by the first system are mined from the training data. The CRF-based classification does not require any explicit linguistic rules but it uses a gazetteer built from Wiki and other sources.

Survey of Named Entity Recognition Techniques for Various Indian Regional Languages

“…Sujan Kumar Saha, Partha Sarathi Ghosh, Sudeshna Sarkar, and Pabitra Mitra [18], in this paper, the authors have described a Maximum Entropy based NER system for Hindi. They have explored different features applicable for the Hindi NER task.…”

Banner: A Cost-Sensitive Contextualized Model for Bangla Named Entity Recognition

“…Figure 1 provides examples which illustrate the real challenges of Bangla language more precisely. Unlike English, there are minimal indicators for tags like capitalization in Bangla as stated by [17], [42]. For instance, under Capitalization in the figure, the words 'chottogram' and 'kushtia' refer to Chittagong and Kushtia, which are names of places.…”

“…Though modern Neural Networks do not rely on capitalization anymore, we still find this problem valid for Bangla as there is no indicator. One of the major characteristics of Bangla is that same word may refer to multiple meaning based on the position of the word [26], [42]. In the figure under Multiple Meaning, both sentences have the same highlighted Bangla word 'kobita' which refers to Kobita as Person in the first sentence and Poem as Object in the second sentence.…”

“…It is possible to detect both entities in English as they are different words, but in Bangla it becomes more complex since the particular word is used differently in the sentences. Furthermore, as Bangla is a relatively free word order language, entities can appear in any position without changing the meaning of a sentence [17], [42]. In the figure under Sentence Structure, the highlighted word 'fulta', which refers to flower is in different positions.…”

“…Moreover, Bangla has highly inflected words. Based on the context of sentences, various prepositions are added with proper nouns [17], [26], [42]. For instance, sentences under Inflection in the figure, the word Dhaka is referred to different word forms in Bangla where both of the words signify Location.…”