Technical Domain Classification of Bangla Text using BERT

Abstract

Coarse-grained tasks are primarily based on Text classification, one of the earliest problems in NLP, and these tasks are done on document and sentence levels. Here, our goal is to identify the technical domain of a given Bangla text. In Coarse-grained technical domain classification, such a piece of the Bangla text provides information about specific Coarse-grained technical domains like Biochemistry (bioche), Communication Technology (com-tech), Computer Science (cse), Management (mgmt), Physics (phy) Etc. This paper uses a recent deep learning model called the Bangla Bidirectional Encoder Representations Transformers (Bangla BERT) mechanism to identify the domain of a given text. Bangla BERT (Bangla-Bert-Base) is a pretrained language model of the Bangla language. Later, we discuss the Bangla BERT accuracy and compare it with other models that solve the same problem.

Aspect-based Sentiment Analysis for Bengali Text using Bidirectional Encoder Representations from Transformers (BERT)

“…BERT includes two steps, pre-training and fine-tuning. • Masked Language Modeling (MLM): The first step is to arbitrarily mask out 15% words in the input text data and replace them with [MASK] tokens [26]. The BERT attention-based encoder is then applied to the entire sequence, which only predicts the masked words based on the context that the other, non-masked words in the sequence provided.…”

“…• Next Sentence Prediction(NSP): NSP predicts the connection between two sentences [26]. The primary task of NSP is to determine whether the second sentence comes after the first sentence or not.…”

“…Fine Tuning: BERT can be fine-tuned to perform a variety of language tasks. By simply placing a single layer over the base model, we can adjust the base model according to our own dataset [26]. The pre-trained parameters are used to initialize the BERT model at first.…”