A Fusion Model-Based Label Embedding and Self-Interaction Attention for Text Classification

Dong, Yanru; Liu, Peiyu; Zhu, Zhenfang; Wang, Qicai; Zhang, Qiuyue

doi:10.1109/access.2019.2954985

Cited by 36 publications

(20 citation statements)

References 30 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, self-attention mechanism has attracted peopleʼs attention and achieved state-of-the-art results. Dong et al [25] combined a self-interaction attention mechanism with label representation and used the Bert model to solve the problem of text feature extraction. In this method, joint word representation and label representation are proposed to improve the efficiency of the model.…”

Section: Complex Neural Network Methodmentioning

confidence: 99%

Social Network Spam Detection Based on ALBERT and Combination of Bi-LSTM with Self-Attention

Zhou

Liu

2021

Security and Communication Networks

View full text Add to dashboard Cite

Social networks are full of spams and spammers. Although social network platforms have established a variety of strategies to prevent the spread of spam, strict information review mechanism has given birth to smarter spammers who disguise spam as text sent by ordinary users. In response to this, this paper proposes a spam detection method powered by the self-attention Bi-LSTM neural network model combined with ALBERT, a lightweight word vector model of BERT. We take advantage of ALBERT to transform social network text into word vectors and then input them to the Bi-LSTM layer. After feature extraction and combined with the information focus of the self-attention layer, the final feature vector is obtained. Finally, SoftMax classifier performs classification to obtain the result. We verify the excellence of the model with accuracy, precision, F1-score, etc. The results show that the model has better performance than others.

show abstract

Section: Complex Neural Network Methodmentioning

confidence: 99%

Social Network Spam Detection Based on ALBERT and Combination of Bi-LSTM with Self-Attention

Zhou

Liu

2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Word2vec s high-dimensional vector model solves the multi-dimensional semantic problem, because it can quickly and effectively express words in high-dimensional vector form through the optimized training model according to a given corpus, thereby providing a new tool for the application research in the field of natural language processing [63]. Academic research [64,65] demonstrates that Word2vec has achieved excellent performance in the fields of text similarity calculation and text classification. In light of the above analysis, this study opted to construct Word2vec vectors for the pre-processed and semantically expanded comment text.…”

Section: Text Representation (1) Text Representation Of Comments Based On Word2vecmentioning

confidence: 99%

Research on Sentiment Classification of Online Travel Review Text

Wen

Zheng

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

In recent years, the number of review texts on online travel review sites has increased dramatically, which has provided a novel source of data for travel research. Sentiment analysis is a process that can extract tourists’ sentiments regarding travel destinations from online travel review texts. The results of sentiment analysis form an important basis for tourism decision making. Thus far, there has been minimal concern as to how sentiment analysis methods can be effectively applied to improve the effect of sentiment analysis. However, online travel review texts are largely short texts characterized by uneven sentiment distribution, which makes it difficult to obtain accurate sentiment analysis results. Accordingly, in order to improve the sentiment classification accuracy of online travel review texts, this study transformed sentiment analysis into a multi-classification problem based on machine learning methods, and further designed a keyword semantic expansion method based on a knowledge graph. Our proposed method extracts keywords from online travel review texts and obtains the concept list of keywords through Microsoft Knowledge Graph. This list is then added to the review text to facilitate the construction of semantically expanded classification data. Our proposed method increases the number of classification features used for short text by employing the huge corpus of information associated with the knowledge graph. In addition, this article introduces online travel review text preprocessing, keyword extraction, text representation, sampling, establishment classification labeling, and the selection and application of machine learning-based sentiment classification methods in order to build an effective sentiment classification model for online travel review text. Experiments were implemented and evaluated based on the English review texts of four famous attractions in four countries on the TripAdvisor website. Our experimental results demonstrate that the method proposed in this paper can be used to effectively improve the accuracy of the sentiment classification of online travel review texts. Our research attempts to emphasize and improve the methodological relevance and applicability of sentiment analysis for future travel research.

show abstract

“…The context vector c is calculated with these attention weights, and inputted to the decoder, which makes the decoder can access the whole input sequence also focus on relevant positions in the input sequence. Given good performance, there are numerous applications of AM in Natural Language Processing [25], [26], Speech Recognition [27], [28], Text Processing [29] and Computer Vision [30], et al…”

Section: Attention Mechanismmentioning

confidence: 99%

High Accuracy Individual Identification Model of Crested Ibis (Nipponia Nippon) Based on Autoencoder With Self-Attention

et al. 2020

View full text Add to dashboard Cite

As the population and the distribution of Crested Ibis (Nipponia nippon) become larger, it is necessary to propose a highly efficient census method to estimate the population size of the Crested Ibis. Passive acoustic monitoring (PAM) has a very good prospect for the Crested Ibis monitoring. To realize the automatic census of the Crested Ibis with PAM, the automatic individual identification method based on the vocalization is the key technology. A novel individual identification model was proposed in this paper, which built the autoencoder based on LSTM to obtain the meaningful latent representation from the raw recording directly, further, embedded self-attention and putted forward a combined training mode to achieve distinctive latent representation. With this model, nine Crested Ibis individuals were identified accurately, the highest accuracy is 0.971, and the average accuracy reaches 0.958. As for other three species, Little owl (Athene noctua), Chiffchaff (Phylloscopus collybita) and Tree pipit (Anthus trivialis), the better performances were achieved than the existing method, which means the proposed model can provide an alternative method for the individual identification of other bird species.

show abstract

A Fusion Model-Based Label Embedding and Self-Interaction Attention for Text Classification

Cited by 36 publications

References 30 publications

Social Network Spam Detection Based on ALBERT and Combination of Bi-LSTM with Self-Attention

Social Network Spam Detection Based on ALBERT and Combination of Bi-LSTM with Self-Attention

Research on Sentiment Classification of Online Travel Review Text

High Accuracy Individual Identification Model of Crested Ibis (Nipponia Nippon) Based on Autoencoder With Self-Attention

Contact Info

Product

Resources

About