Subsentence Extraction from Text Using Coverage-Based Deep Learning Language Models

Abstract

Sentiment prediction remains a challenging and unresolved task in various research fields, including psychology, neuroscience, and computer science. This stems from its high degree of subjectivity and limited input sources that can effectively capture the actual sentiment. This can be even more challenging with only text-based input. Meanwhile, the rise of deep learning and an unprecedented large volume of data have paved the way for artificial intelligence to perform impressively accurate predictions or even human-level reasoning. Drawing inspiration from this, we propose a coverage-based sentiment and subsentence extraction system that estimates a span of input text and recursively feeds this information back to the networks. The predicted subsentence consists of auxiliary information expressing a sentiment. This is an important building block for enabling vivid and epic sentiment delivery (within the scope of this paper) and for other natural language processing tasks such as text summarisation and Q&A. Our approach outperforms the state-of-the-art approaches by a large margin in subsentence prediction (i.e., Average Jaccard scores from 0.72 to 0.89). For the evaluation, we designed rigorous experiments consisting of 24 ablation studies. Finally, our learned lessons are returned to the community by sharing software packages and a public dataset that can reproduce the results presented in this paper.

Transformer transfer learning emotion detection model: synchronizing socially agreed and self-reported emotions in big data

“…This version uses the same architecture as BERT but pretrains ten times more data, including both BERT data and 63 million news articles, a web text corpus, and stories. Because it offers the highest level of forecasting accuracy to date, we apply RoBERTa-large to compare emotion detection models [ 19 ].…”

“…With RoBERTa, a fine-tuning process takes place for sequence-level classification tasks, as in the BERT architecture. We follow an existing process [ 19 ], in which emotion data sets get split into a training (80%) and a testing (20%) data set for the transformer model training. At the beginning of the sequence, a special class token [CLS] gets added for classification tasks.…”

“…3 ). During training, we fine-tuned all parameters of the transformer model together, and the output label probabilities were calculated using the Adam function [ 19 ]. This process appears as the middle area in Fig.…”

Applying Few-Shot Learning in Classifying Pedestrian Crash Typing

“…Figure 2 illustrates the overall architecture of the BERT classification network. The input text is split into a sequence of tokens (Tok) and then processed in the BERT model ( 22 ). [CLS] is a special classification token inserted at the beginning of every input sequence.…”

“…Figure 2 illustrates the overall architecture of the BERT classification network. The input text is split into a sequence of tokens (Tok) and then processed in the BERT model (22).…”