Explainable Sentiment Analysis: A Hierarchical Transformer-Based Extractive Summarization Approach

Abstract

In recent years, the explainable artificial intelligence (XAI) paradigm is gaining wide research interest. The natural language processing (NLP) community is also approaching the shift of paradigm: building a suite of models that provide an explanation of the decision on some main task, without affecting the performances. It is not an easy job for sure, especially when very poorly interpretable models are involved, like the almost ubiquitous (at least in the NLP literature of the last years) transformers. Here, we propose two different transformer-based methodologies exploiting the inner hierarchy of the documents to perform a sentiment analysis task while extracting the most important (with regards to the model decision) sentences to build a summary as the explanation of the output. For the first architecture, we placed two transformers in cascade and leveraged the attention weights of the second one to build the summary. For the other architecture, we employed a single transformer to classify the single sentences in the document and then combine the probability scores of each to perform the classification and then build the summary. We compared the two methodologies by using the IMDB dataset, both in terms of classification and explainability performances. To assess the explainability part, we propose two kinds of metrics, based on benchmarking the models’ summaries with human annotations. We recruited four independent operators to annotate few documents retrieved from the original dataset. Furthermore, we conducted an ablation study to highlight how implementing some strategies leads to important improvements on the explainability performance of the cascade transformers model.

Transformer based Contextual Model for Sentiment Analysis of Customer Reviews: A Fine-tuned BERT

“…www.ijacsa.thesai.org In 2021, [11] conducted an Aspect-based SA study on consumers product reviews data. They have proposed two BERT models for aspect extraction, sentiment classification using parallel and hierarchical aggregation methods based on hierarchical transformer model [12]. The following Fig.…”

“…The BERT BASE and BERT LARGE are two original models. The base model consists of 12 Encoders and bidirectional self-attention, while the large model consists of 24 Encoders and 16 bidirectional heads. BERT model is pre-trained on 800 million words from BooksCorpus and English Wikipedia's unlabeled text of 2.5 billion words.…”

Towards the Generation of Musical Explanations with GPT-3

“…Some output explanations can be seen in table 7. Detailed transcripts from GPT-3 can be found here 4 . Generally, GPT-3 provided poor explanations.…”

“…Although these explanations provide insights about the decisions behind the model, they fail to provide enough semantic meaning with respect to the creative process. In [4], an attempt is made to provide deeper explanations in the task of sentiment classification. This also involved an analysis of the attention weights; however, instead of focusing the explanations on this feature, they provided a summary of the most relevant sentences (according to the weights) as the explanation.…”

“…The AI musician has recently finished composing a piece of music and is excited to talk about and explain various aspects about their song". Questions and answers relating to this prompt can be seen in table 4.…”