A Comprehensive Survey on Word Representation Models: From Classical to State-of-the-Art Word Representation Language Models

Naseem, Usman; Razzak, Imran; Khan, Shah Khalid; Prasad, Mukesh

doi:10.1145/3434237

Cited by 83 publications

(40 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Natural language processing (NLP) has flourished over the past couple of years as a result of advancements in deep learning [263]. One important component of NLP is building language models [252]. Today, deep neural networks using transformers [325] are utilized to construct language models [111,131] for a variety of tasks.…”

Section: Natural Language Modelingmentioning

confidence: 99%

A Survey of Quantum Computing for Finance

Herman¹,

Googin²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Quantum computers are expected to surpass the computational capabilities of classical computers during this decade and have transformative impact on numerous industry sectors, particularly finance. In fact, finance is estimated to be the first industry sector to benefit from quantum computing, not only in the medium and long terms, but even in the short term. This survey paper presents a comprehensive summary of the state of the art of quantum computing for financial applications, with particular emphasis on Monte Carlo integration, optimization, and machine learning, showing how these solutions, adapted to work on a quantum computer, can help solve more efficiently and accurately problems such as derivative pricing, risk analysis, portfolio optimization, natural language processing, and fraud detection. We also discuss the feasibility of these algorithms on near-term quantum computers with various hardware implementations and demonstrate how they relate to a wide range of use cases in finance. We hope this article will not only serve as a reference for academic researchers and industry practitioners but also inspire new ideas for future research.

show abstract

Section: Natural Language Modelingmentioning

confidence: 99%

A Survey of Quantum Computing for Finance

Herman¹,

Googin²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For addressing this problem, many works have used feature selection techniques [25,26] applying various machine learning approaches. Of the various machine learning classification methods used to classify users' sentiments from a text, decision tree, LDA, Naive Bayes, Support Vector Machine (SVM), and artificial neural networks are the most common and have achieved a higher performance [9,22,27,28]. However, these methods need massive training data and are often slow.…”

Section: Sentiment Analysismentioning

confidence: 99%

“…In most natural language processing applications, words are used as features. The most popular word vector representations are distributed representation and one-hot representation [27,47]. However, the one-hot representation has various problems, such as the too-large vector dimension, the sparsity of the word vector, and ignoring the word semantic association.…”

Section: Embedding (Word Representation)mentioning

confidence: 99%

Transformer-Based Graph Convolutional Network for Sentiment Analysis

et al. 2022

View full text Add to dashboard Cite

Sentiment Analysis is an essential research topic in the field of natural language processing (NLP) and has attracted the attention of many researchers in the last few years. Recently, deep neural network (DNN) models have been used for sentiment analysis tasks, achieving promising results. Although these models can analyze sequences of arbitrary length, utilizing them in the feature extraction layer of a DNN increases the dimensionality of the feature space. More recently, graph neural networks (GNNs) have achieved a promising performance in different NLP tasks. However, previous models cannot be transferred to a large corpus and neglect the heterogeneity of textual graphs. To overcome these difficulties, we propose a new Transformer-based graph convolutional network for heterogeneous graphs called Sentiment Transformer Graph Convolutional Network (ST-GCN). To the best of our knowledge, this is the first study to model the sentiment corpus as a heterogeneous graph and learn document and word embeddings using the proposed sentiment graph transformer neural network. In addition, our model offers an easy mechanism to fuse node positional information for graph datasets using Laplacian eigenvectors. Extensive experiments on four standard datasets show that our model outperforms the existing state-of-the-art models.

show abstract

“…The authors of [5] conducted a rich survey of language representation models, including many models from 2020 and 2021. They additionally provided useful insights about pre-processing, classification techniques, evaluation metrics, and applications.…”

Section: Related Workmentioning

confidence: 99%

Language Representation Models: An Overview

Schomacker

Tropmann-Frick

2021

Entropy

View full text Add to dashboard Cite

In the last few decades, text mining has been used to extract knowledge from free texts. Applying neural networks and deep learning to natural language processing (NLP) tasks has led to many accomplishments for real-world language problems over the years. The developments of the last five years have resulted in techniques that have allowed for the practical application of transfer learning in NLP. The advances in the field have been substantial, and the milestone of outperforming human baseline performance based on the general language understanding evaluation has been achieved. This paper implements a targeted literature review to outline, describe, explain, and put into context the crucial techniques that helped achieve this milestone. The research presented here is a targeted review of neural language models that present vital steps towards a general language representation model.

show abstract

A Comprehensive Survey on Word Representation Models: From Classical to State-of-the-Art Word Representation Language Models

Cited by 83 publications

References 95 publications

A Survey of Quantum Computing for Finance

A Survey of Quantum Computing for Finance

Transformer-Based Graph Convolutional Network for Sentiment Analysis

Language Representation Models: An Overview

Contact Info

Product

Resources

About