Vector representation based on a supervised codebook for Nepali documents classification

Abstract

Document representation with outlier tokens exacerbates the classification performance due to the uncertain orientation of such tokens. Most existing document representation methods in different languages including Nepali mostly ignore the strategies to filter them out from documents before learning their representations. In this article, we propose a novel document representation method based on a supervised codebook to represent the Nepali documents, where our codebook contains only semantic tokens without outliers. Our codebook is domain-specific as it is based on tokens in a given corpus that have higher similarities with the class labels in the corpus. Our method adopts a simple yet prominent representation method for each word, called probability-based word embedding. To show the efficacy of our method, we evaluate its performance in the document classification task using Support Vector Machine and validate against widely used document representation methods such as Bag of Words, Latent Dirichlet allocation, Long Short-Term Memory, Word2Vec, Bidirectional Encoder Representations from Transformers and so on, using four Nepali text datasets (we denote them shortly as A1, A2, A3 and A4). The experimental results show that our method produces state-of-the-art classification performance (77.46% accuracy on A1, 67.53% accuracy on A2, 80.54% accuracy on A3 and 89.58% accuracy on A4) compared to the widely used existing document representation methods. It yields the best classification accuracy on three datasets (A1, A2 and A3) and a comparable accuracy on the fourth dataset (A4). Furthermore, we introduce the largest Nepali document dataset (A4), called NepaliLinguistic dataset, to the linguistic community.

Deep Learning‐Based Methods for Sentiment Analysis on Nepali COVID‐19‐Related Tweets

“…First, most of the existing works [ 2 , 4 , 5 , 12 ] on COVID-19-related tweets are performed in high-resource languages such as English and Arabic. The approach used by high-resource language might be inapplicable to low-resource languages such as Nepali, which is based on Devanagari script and has 36 consonants (33 are distinct consonants and 3 are combined consonants), 13 vowels, and 10 numerals ( Figure 1 ) [ 1 , 15 , 16 ]. Second, their investigation mainly targets either clustering the tweets into various themes/topics or classifying their polarity into three classes (negative, positive, or neutral) using the well-established feature extraction methods such as BERT, Word2Vec, and Glove.…”

“…For example, semantic features based on the COVID-19-related tweets could learn more informative features. For this, we employ the probabilistic feature extraction approach as suggested by Sitaula et al [ 1 ] recently, which calculates the probability of each input word across all categories and finally and attains the feature vector depending on the number of categories present in the dataset. Last, with the help of the domain-agnostic method, we capture the semantic information using the cross-domain approach, which means that we transfer the knowledge to current COVID-19 domain from another domain such as news categories.…”

Automated Amharic News Categorization Using Deep Learning Models

“…During word representation learning, fastText considers not only the word itself but also groups of characters from that word and subword information such as character unigrams, bigrams, and trigrams [ 20 ]. However, GloVe and word2vec fail to provide any vector representation for words that are not in the model dictionary [ 21 ]. As a result, in this study, fastText is used as a word representation model.…”

Convolutional neural network-based ensemble methods to recognize Bangla handwritten character

“…Various research have also been conducted for the recognition of handwritten characters and texts from other languages such as recognizing Baybayin scripts using SVM ( Sitaula, Basnet & Aryal, 2021 ), analyzing handwritten Hebrew document ( Biller et al, 2016 ), recognizing English handwritings ( Pham et al, 2020 ), and many more. For English handwritten digit and character recognition tasks, CNN-based architectures have yielded better performance than other techniques ( Baldominos, Saez & Isasi, 2019 ; Ranzato et al, 2007 ; Cireşan et al, 2011 ), and so on.…”