Exploiting BERT for End-to-End Aspect-based Sentiment Analysis

Li, Xin; Bing, Lidong; Zhang, Wenxuan; Lam, Wai

doi:10.18653/v1/d19-5505

Cited by 236 publications

(111 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Biomedical Named Entity Recognition is a challenging task in the field of biomedical information processing. Li et al [31] proposed the use of BERT embeddings and Conditional Random Fields (CRF) layer for finding the best tag sequence for a given sentence. Yuan [51] proposed a BERT-based question answering model, where they used a BERT-CRF model in order to detect mentions of the entities in sentences.…”

Section: Contextualized Embeddings Based Modelsmentioning

confidence: 99%

“…On the other hand, many recent state-of-the-art architectures used CRF layer after a contextual Language Model [31], [45], [49], [51]. Both Li [31] and Yuan et al [51] used BERT-CRF model for sequence labelling tasks. In [45], Souza et al employed the BERT model with CRF layer using feature-based and fine-tuning based strategies for Portuguese NER task.…”

Section: F Conditional Random Fieldmentioning

confidence: 99%

“…In many state-of-the-art architectures, Conditional Random Field (CRF) was widely used for sequence labelling tasks. [2], [28], [31], [35], [45], [51].…”

Section: Integration Of Crf Layermentioning

confidence: 99%

See 2 more Smart Citations

Banner: A Cost-Sensitive Contextualized Model for Bangla Named Entity Recognition

Ashrafi¹,

Mohammad²,

Mauree³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Named Entity Recognition (NER) is a task in Natural Language Processing (NLP) that aims to classify words into a predetermined list of Named Entities (NE). Many architectures have produced good results on high resourced languages like English and Chinese. However, the NER task has not yet achieved much progress for Bangla, a low resource Language. In this paper, we perform the NER task on Bangla Language using Word2Vec and contextual Bidirectional Encoder Representations from Transformers (BERT) embeddings. We propose multiple BERT-based deep learning models that use the contextualized embedding from BERT as inputs and a simple statistical approach for class weight cost sensitive learning. The modified cost-sensitive loss function was used to address the class imbalance of the data. In our modified cost-sensitive loss function, we penalize the dominant classes by taking the ratio concerning the maximum sample in a class instead of the whole dataset. This penalty is made so that the learner learns slowly for the dominant class. In addition, we experiment by adding a Conditional Random Field (CRF) layer and incorporating Focal Loss to the training process. We found the best F1 Macro score to be 65.96%, F1 Micro score of 90.64%, and F1 Message Understanding Coreference (MUC) score of 72.04%, which were calculated at Named Entity level. Our experimental results demonstrate that one of the proposed models, which jointly optimizes for the CRF loss and class weighted cost-sensitive loss according to our proposed statistical approach, achieve an improvement of over 8% F1 MUC score on a recently introduced Bangla NER dataset when compared to previously published work.

show abstract

Section: Contextualized Embeddings Based Modelsmentioning

confidence: 99%

Section: F Conditional Random Fieldmentioning

confidence: 99%

See 1 more Smart Citation

Banner: A Cost-Sensitive Contextualized Model for Bangla Named Entity Recognition

Ashrafi¹,

Mohammad²,

Mauree³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The advantage of such an approach is that less parameters need to be learned from scratch. Several works have shown that BERT can be transferred for tasks, such as text summarization and sentiment analysis (Li, Bing, Zhang, & Lam, 2019; Liu & Lapata, 2019). Biomedical NLP researchers have also demonstrated the importance of transfer learning from pre‐trained BERT, where the state‐of‐the‐art performances are obtained by fine‐tuning BERT with large task‐/domain‐specific data in NER, question answering (QA) and relation extraction (Lee et al, 2020).…”

Section: Word Representation Modelsmentioning

confidence: 99%

Word embeddings for biomedical natural language processing: A survey

Chiu

Baker

2020

Language and Linguist. Compass

View full text Add to dashboard Cite

Word representations are mathematical objects that capture the semantic and syntactic properties of words in a way that is interpretable by machines. Recently, encoding word properties into low‐dimensional vector spaces using neural networks has become increasingly popular. Word embeddings are now used as the main input to natural language processing (NLP) applications, achieving cutting‐edge results. Nevertheless, most word‐embedding studies are carried out with general‐domain text and evaluation datasets, and their results do not necessarily apply to text from other domains (e.g., biomedicine) that are linguistically distinct from general English. To achieve maximum benefit when using word embeddings for biomedical NLP tasks, they need to be induced and evaluated using in‐domain resources. Thus, it is essential to create a detailed review of biomedical embeddings that can be used as a reference for researchers to train in‐domain models. In this paper, we review biomedical word embedding studies from three key aspects: the corpora, models and evaluation methods. We first describe the characteristics of various biomedical corpora, and then compare popular embedding models. After that, we discuss different evaluation methods for biomedical embeddings. For each aspect, we summarize the various challenges discussed in the literature. Finally, we conclude the paper by proposing future directions that will help advance research into biomedical embeddings.

show abstract

“…More recently, some scholars even propose complicated models with the help of more advanced neural structures, e.g., transformation networks [12], parameterized convolutional neural networks (PCNN) [13], gated convolutional networks (GCN) [14], memory networks [15], graph networks [16], [17] or semantic cognition networks (SCN) [18]. However, just as Li et al pointed out in [19], the improvement of these models measured by the accuracy or F1 score has reached a bottleneck, because the commonly used embeddings are pre-trained via word2vec or GloVe, which can only provide context-independent word-level features and are insufficient for capturing the complex semantic dependencies in the sentence.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Aspect-Based Sentiment Analysis With Capsule Network

Luo

2020

IEEE Access

View full text Add to dashboard Cite

Existing feature-based neural approaches for aspect-based sentiment analysis (ABSA) try to improve their performance with pre-trained word embeddings and by modeling the relations between the text sequence and the aspect (or category), thus heavily depending on the quality of word embeddings and task-specific architectures. Although the recently pre-trained language models, i.e., BERT and XLNet, have achieved state-of-the-art performance in a variety of natural language processing (NLP) tasks, they still subject to the aspect-specific, local feature-aware and task-agnostic challenges. To address these challenges, this paper proposes a XLNet and capsule network based model XLNetCN for ABSA. XLNetCN firstly constructs auxiliary sentence to model the sequence-aspect relation and generate global aspect-specific representations, which enables to enhance aspect-awareness and ensure the full pre-training of XLNet for improving task-agnostic capability. After that, XLNetCN also employs a capsule network with the dynamic routing algorithm to extract the local and spatial hierarchical relations of the text sequence, and yield its local feature representations, which are then merged with the global aspect-related representations for downstream classification via a softmax classifier. Experimental results show that XLNetCN outperforms significantly than the classical BERT, XLNet and traditional feature-based approaches on the two benchmark datasets of SemEval 2014, Laptop and Restaurant, and achieves new state-of-the-art results. INDEX TERMS Aspect-based sentiment analysis, natural language processing, text analysis, deep learning, neural network.

show abstract

Exploiting BERT for End-to-End Aspect-based Sentiment Analysis

Cited by 236 publications

References 53 publications

Banner: A Cost-Sensitive Contextualized Model for Bangla Named Entity Recognition

Banner: A Cost-Sensitive Contextualized Model for Bangla Named Entity Recognition

Word embeddings for biomedical natural language processing: A survey

Enhancing Aspect-Based Sentiment Analysis With Capsule Network

Contact Info

Product

Resources

About