Arabic Gloss WSD Using BERT

Abstract

Word Sense Disambiguation (WSD) aims to predict the correct sense of a word given its context. This problem is of extreme importance in Arabic, as written words can be highly ambiguous; 43% of diacritized words have multiple interpretations and the percentage increases to 72% for non-diacritized words. Nevertheless, most Arabic written text does not have diacritical marks. Gloss-based WSD methods measure the semantic similarity or the overlap between the context of a target word that needs to be disambiguated and the dictionary definition of that word (gloss of the word). Arabic gloss WSD suffers from a lack of context-gloss datasets. In this paper, we present an Arabic gloss-based WSD technique. We utilize the celebrated Bidirectional Encoder Representation from Transformers (BERT) to build two models that can efficiently perform Arabic WSD. These models can be trained with few training samples since they utilize BERT models that were pretrained on a large Arabic corpus. Our experimental results show that our models outperform two of the most recent gloss-based WSDs when we test them against the same test data used to evaluate our model. Additionally, our model achieves an F1-score of 89% compared to the best-reported F1-score of 85% for knowledge-based Arabic WSD. Another contribution of this paper is introducing a context-gloss benchmark that may help to overcome the lack of a standardized benchmark for Arabic gloss-based WSD.

ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD

“…In this paper, we investigate the use of different types of signals to emphasize target words in context for Arabic WSD. El-Razzaz et al (2021) fine-tuned two BERT models on a small dataset of context-gloss pairs, consisting of about 5k lemmas, about 15k positive and 15k negative context-gloss pairs. They claimed an F1-score of 89%.…”

Multimodal deep learning model for human handover classification

“…On the other hand, FP (false positive) count represents the number of class 2 samples that have been miss-classified as class 1 and vice-versa with FN (false negative) count. A specific part in the work done by El-Razzaz explains these metrics briefly and utilizes it in other evaluations [29].…”

Context-Gloss Augmentation for Improving Arabic Target Sense Verification

“…TSV has proven to be an effective solution for the WSD in many state-of-the-art efforts. Although some researchers did not use the term TSV, this notion was also referred to as GlossBERT or Context-Gloss Binary Classification (Al-Hajj and Jarrar, 2022;El-Razzaz et al, 2021). A TSV training dataset is typically a set of context-gloss pairs, each labeled with Positive or Negative.…”

“…It can be fine-tuned on domain/task-specific data (e.g., POS tagging, WSD, TSV, and WiC) to update its contextualized embeddings. The TSV task has been addressed by fine-tuning BERT on context-gloss pairs as a sentence pair binary classification problem (Huang et al, 2019;Yap et al, 2020;Patel et al, 2021;Ranjbar and Zeinali, 2021;Lin and Giambi, 2021;El-Razzaz et al, 2021;Al-Hajj and Jarrar, 2022). However, the TSV task, similar to most NLP tasks, suffers from the knowledge-gain bottleneck, i.e., the lack of available quality datasets to train machine learning models.…”