BERT-Based Sentiment Analysis for Low-Resourced Languages: A Case Study of Urdu Language

Abstract

Sentiment analysis holds significant importance in research projects by providing valuable insights into public opinions. However, the majority of sentiment analysis studies focus on the English language, leaving a gap in research for other low-resourced languages or regional languages, e.g., Persian, Pashto, and Urdu. Moreover, computational linguists face the challenge of developing lexical resources for these languages. In light of this, this paper presents a deep learningbased approach for Urdu Text Sentiment Analysis (USA-BERT), leveraging Bidirectional Encoder Representations from Transformers and introduces an Urdu Dataset for Sentiment Analysis-23 (UDSA-23). USA-BERT first preprocesses the Urdu reviews by exploiting BERT-Tokenizer. Second, it creates BERT embeddings for each Urdu review. Third, given the BERT embeddings, it fine-tunes a deep learning classifier (BERT). Finally, it employs the Pareto principle on two datasets (the state-of-the-art (UCSA-21) and UDSA-23) to assess USA-BERT. The assessment results demonstrate that USA-BERT significantly surpasses the existing methods by improving the accuracy and f-measure up to 26.09% and 25.87%, respectively.

Sentiment Analysis in Low-Resource Settings: A Comprehensive Review of Approaches, Languages, and Data Sources

“…Another research by [117], delves into a deep learningbased methodology for sentiment analysis in Urdu, utilizing BERT transformers. The study introduces a novel Urdu Sentiment Analysis Dataset-23 tailored for computational linguists.…”

“…However, word embedding, harnesses distributed representations of words but struggles with issues of polysemy and time consumption. On the other hand, the transfer learning approach, offers promise in sentiment analysis classification using pre-trained models, notably transformer architectures, as exemplified by [94], [105], [111], [117], which excel in capturing contextual information. Figure 8 illustrates the selected studies', approaches.…”

“…Nonetheless, challenges persist, particularly in accurately representing all languages within tokenizer frameworks due to variations in language characters and other linguistic factors. Data cleaning [95], [35], [96], [49], [62], [65], [58], [68], [67], [89], [74], [75], [76], [77], [88], [94], [100], [103], [104], [107], [80], [106], [81], [108], [109], [82], [53], [110], [111], [83], [92], [113], [114], [115], [116], [117] Stemming [51], [67], [104], [108],…”