A neural topic model with word vectors and entity vectors for short texts

Zhao, Xiaowei; Wang, Deqing; Zhao, Zhengyang; Liu, Wei; Lu, Chenwei; Zhuang, Fuzhen

doi:10.1016/j.ipm.2020.102455

Cited by 41 publications

(15 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Word vectors are all words, so word vectors can form document vectors. In general, the training of the word vector is carried out first, and then the word vector is expressed as a text vector by means of summation [21]. e word2vec model using skip-gram is used for text representation.…”

Section: Construction Of Forecasting Model Using Financial Datamentioning

confidence: 99%

[Retracted] Analysis and Risk Assessment of Corporate Financial Leverage Using Mobile Payment in the Era of Digital Technology in a Complex Environment

Wei

2022

Journal of Mathematics

View full text Add to dashboard Cite

The study aims to improve the enterprise’s ability to respond to financial crises and find some countermeasures to prevent potential financial risks. The enterprise financial risk is assessed, and the automatic summary function of mobile payment platforms based on long short-term memory (LSTM) is performed to extract the structured data and unstructured texts from its annual report. On this basis, the early warning system model of financial risks is implemented and its accuracy is improved. The structured data and unstructured text in the company’s annual report are extracted. The enterprise financial risk early warning system model is constructed. The accuracy of the enterprise financial risk early warning system has been improved. Firstly, we use the convolutional neural network (CNN) to establish a financial risk prediction system using financial data and test various indicators of the system. Secondly, the financial annual report of the listed company is obtained from the Internet. The required financial statements are obtained in two ways. The first is to set high special treatment (ST) sample weights and delete some non-ST samples. The second is to delete punctuation marks, interjections, numbers, and so on and process the collected text data. The financial risk prediction model is established using the financial text, and the LSTM + attention mechanism is used to optimize the model. Finally, combining structured financial data and unstructured financial text to establish a forecasting model, the model uses LSTM. Combined with a single-layer neural network or CNN model, the comparison experiment is carried out in two ways. Experiments show that the CNN or LSTM attention mechanism cannot significantly improve the performance of the system only using financial data or texts. Using the financial data and financial text using the LSTM + CNN model, the F1 value reached 85.29%. Financial data and other indicators in the text have also been greatly improved, and the overall performance is the best. In summary, LSTM using financial data and financial texts combined with CNN to establish a risk prediction system can help investors and companies themselves find possible financial crises in listed companies as soon as possible and help companies deal with their financial risks in a timely manner.

show abstract

Section: Construction Of Forecasting Model Using Financial Datamentioning

confidence: 99%

[Retracted] Analysis and Risk Assessment of Corporate Financial Leverage Using Mobile Payment in the Era of Digital Technology in a Complex Environment

Wei

2022

Journal of Mathematics

View full text Add to dashboard Cite

show abstract

“…The authors test on the Twenty Newsgroups and a NIPS data set. Also in 2021, Zhao et al introduced another neural topic model that incorporates word embedding vectors and entity vectors into the model [96] (VAETM). Word vectors have been incorporated in both neural and unsupervised topic models.…”

Section: Meta-data Augmented Supervised and Reinforcement Learning Ba...mentioning

confidence: 99%

The Evolution of Topic Modeling

Churchill

Singh

2022

ACM Comput. Surv.

View full text Add to dashboard Cite

Topic models have been applied to everything from books to newspapers to social media posts in an effort to identify the most prevalent themes of a text corpus. We provide an in-depth analysis of unsupervised topic models from their inception to today. We trace the origins of different types of contemporary topic models, beginning in the 1990s, and we compare their proposed algorithms, as well as their different evaluation approaches. Throughout, we also describe settings in which topic models have worked well and areas where new research is needed, setting the stage for the next generation of topic models.

show abstract

“…In [ 46 ], the authors aggregated short texts into long documents and incorporated document embedding to provide word co-occurrence information. In [ 47 ], a variational autoencoder topic model (VAETM) and a supervised version (SVAETM) of it have been proposed by combining embedded representations of words and entities by employing an external corpus. To enhance contextual information, the authors in [ 48 ] proposed a graph neural network as the encoder of NTM, which accepts a bi-term graph of the words as inputs and produces the topic distribution of the corpus as the output.…”

Section: Neural-topic Models and Related Workmentioning

confidence: 99%

Investigating the Efficient Use of Word Embedding with Neural-Topic Models for Interpretable Topics from Short Texts

Murakami

Chakraborty

2022

Sensors

View full text Add to dashboard Cite

With the rapid proliferation of social networking sites (SNS), automatic topic extraction from various text messages posted on SNS are becoming an important source of information for understanding current social trends or needs. Latent Dirichlet Allocation (LDA), a probabilistic generative model, is one of the popular topic models in the area of Natural Language Processing (NLP) and has been widely used in information retrieval, topic extraction, and document analysis. Unlike long texts from formal documents, messages on SNS are generally short. Traditional topic models such as LDA or pLSA (probabilistic latent semantic analysis) suffer performance degradation for short-text analysis due to a lack of word co-occurrence information in each short text. To cope with this problem, various techniques are evolving for interpretable topic modeling for short texts, pretrained word embedding with an external corpus combined with topic models is one of them. Due to recent developments of deep neural networks (DNN) and deep generative models, neural-topic models (NTM) are emerging to achieve flexibility and high performance in topic modeling. However, there are very few research works on neural-topic models with pretrained word embedding for generating high-quality topics from short texts. In this work, in addition to pretrained word embedding, a fine-tuning stage with an original corpus is proposed for training neural-topic models in order to generate semantically coherent, corpus-specific topics. An extensive study with eight neural-topic models has been completed to check the effectiveness of additional fine-tuning and pretrained word embedding in generating interpretable topics by simulation experiments with several benchmark datasets. The extracted topics are evaluated by different metrics of topic coherence and topic diversity. We have also studied the performance of the models in classification and clustering tasks. Our study concludes that though auxiliary word embedding with a large external corpus improves the topic coherency of short texts, an additional fine-tuning stage is needed for generating more corpus-specific topics from short-text data.

show abstract

A neural topic model with word vectors and entity vectors for short texts

Cited by 41 publications

References 18 publications

[Retracted] Analysis and Risk Assessment of Corporate Financial Leverage Using Mobile Payment in the Era of Digital Technology in a Complex Environment

[Retracted] Analysis and Risk Assessment of Corporate Financial Leverage Using Mobile Payment in the Era of Digital Technology in a Complex Environment

The Evolution of Topic Modeling

Investigating the Efficient Use of Word Embedding with Neural-Topic Models for Interpretable Topics from Short Texts

Contact Info

Product

Resources

About