AgglutiFiT: Efficient Low-Resource Agglutinative Language Model Fine-Tuning

Li, Zhe; Li, Xiuhong; Sheng, Jiabao; Slamu, Wushour

doi:10.1109/access.2020.3015854

Cited by 16 publications

(15 citation statements)

References 15 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Z. Li et al, [101] proposed an efficient strategy that finetunes a pre-trained language model to perform sentiment analysis and text classification called AgglutiFiT. They finetuned the model using the low-noise fine-tuning dataset created by morphological analysis and stem extraction.…”

Section: B Pre-trained (Transformers)mentioning

confidence: 99%

Sentiment Analysis in Low-Resource Settings: A Comprehensive Review of Approaches, Languages, and Data Sources

Aliyu,

Sarlan,

Usman Danyaro

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The field of low-resource sentiment analysis has seen significant developments in recent years. This research review SLR evaluates the approaches and data sources utilized in low-resource sentiment analysis by deep learning. The primary aim is to discover suitable approaches for future sentiment analysis in low-resource. Our studies explore various languages, models, and data sources expressing a desire to create effective approaches. Our emphasis lies in the critical evaluation of the approaches and the datasets utilized, to identify areas where further research is needed. Our analysis study adds to the existing body of literature reviews, encompassing multilingual low-resource sentiment analysis research spanning from 2018 to 2023. The findings indicate that the transfer learning approach is the most frequently used, followed by word embedding learning and machine translation systems. Additionally, the study shows that social media is the most used platform for data collection, followed by product reviews, movies, and hotels. There has been a significant surge in the adoption of pre-trained transformers, indicating a growing interest in exploring the potential of these models for low-resource languages within the natural language processing (NLP) community. This trend is largely attributed to the novel nature of these models and their feature of being nonlabour intensive. However, the scarcity of annotated datasets for such languages remains a major hurdle. finally, these research findings are relevant and informative for any researcher working in the field of lowresource multilingual sentiment analysis. The study introduces a conceptual framework for performing sentiment analysis in low-resource. The study provides a valuable resource for future researchers.

show abstract

Section: B Pre-trained (Transformers)mentioning

confidence: 99%

Sentiment Analysis in Low-Resource Settings: A Comprehensive Review of Approaches, Languages, and Data Sources

Aliyu,

Sarlan,

Usman Danyaro

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…[17][18][19] has carried out a lot of research on cross-domain network structures based on contrastive learning, and applying these methods in the field of aspect-level sentiment analysis is also a great breakthrough; In Ref. [20][21][22], cross-domain network structures are used to conduct a large number of studies on text classification tasks when there are few corpora. As one of the classification tasks, whether aspect-level sentiment analysis can draw lessons from these methods is also worth exploring for subsequent studies.…”

Section: Introductionmentioning

confidence: 99%

GCAT-GTCU: Graph-Connected Attention Network and Gate Than Change Unit for Aspect-Level Sentiment Analysis

Wang

et al. 2023

Symmetry

Self Cite

View full text Add to dashboard Cite

Currently, attention mechanisms are widely used in aspect-level sentiment analysis tasks. Previous studies have only used attention mechanisms combined with neural networks for aspect-level sentiment classification, and the feature extraction of the model is insufficient. When the same aspect and sentiment polarity appear in multiple sentences, the semantic information sharing of the same domain is also ignored, resulting in low model performance. To address these problems, the paper proposes an aspect-level sentiment analysis model, GCAT-GTCU, which combines a Graph-connected Attention Network containing symmetry with Gate Than Change Unit. Three nodes of words, sentences, and aspects are constructed, and local and deep-level features of sentences are extracted using CNN splicing BiGRU; node connection information is added to GAT to form a GCAT containing symmetry to realize the information interaction of three nodes, pay attention to the contextual information, and update the shared information of three nodes at any time; a new gating mechanism GTCU is constructed to filter noisy information and control the flow of sentiment information; finally, the three nodes are extracted information to predict the final sentiment polarity. The experimental results on four publicly available datasets show that the model outperforms the baseline model against which it is compared in some very controlled situations.

show abstract

“…We try to explain this phenomenon. XLM-R may contain more general information in lower layers ( Li et al, 2020 ); While BERT can capture surface features in lower layers, syntactic features in middle layers and semantic features in higher layers ( Jawahar, Sagot & Seddah, 2019 ). Because the surface features of Chinese are not very obvious to be recognized, when mBERT learns the shallow features of Chinese at the lower level, its recognition ability on Chinese is lower than XLM-R’s.…”

Section: Introductionmentioning

confidence: 99%

Probing language identity encoded in pre-trained multilingual models: a typological view

Zheng

Liu

2022

PeerJ Computer Science

View full text Add to dashboard Cite

Pre-trained multilingual models have been extensively used in cross-lingual information processing tasks. Existing work focuses on improving the transferring performance of pre-trained multilingual models but ignores the linguistic properties that models preserve at encoding time—“language identity”. We investigated the capability of state-of-the-art pre-trained multilingual models (mBERT, XLM, XLM-R) to preserve language identity through language typology. We explored model differences and variations in terms of languages, typological features, and internal hidden layers. We found the order of ability in preserving language identity of whole model and each of its hidden layers is: mBERT > XLM-R > XLM. Furthermore, all three models capture morphological, lexical, word order and syntactic features well, but perform poorly on nominal and verbal features. Finally, our results show that the ability of XLM-R and XLM remains stable across layers, but the ability of mBERT fluctuates severely. Our findings summarize the ability of each pre-trained multilingual model and its hidden layer to store language identity and typological features. It provides insights for later researchers in processing cross-lingual information.

show abstract

AgglutiFiT: Efficient Low-Resource Agglutinative Language Model Fine-Tuning

Cited by 16 publications

References 15 publications

Sentiment Analysis in Low-Resource Settings: A Comprehensive Review of Approaches, Languages, and Data Sources

Sentiment Analysis in Low-Resource Settings: A Comprehensive Review of Approaches, Languages, and Data Sources

GCAT-GTCU: Graph-Connected Attention Network and Gate Than Change Unit for Aspect-Level Sentiment Analysis

Probing language identity encoded in pre-trained multilingual models: a typological view

Contact Info

Product

Resources

About