Topically Driven Neural Language Model

Lau, Jey Han; Baldwin, Timothy; Cohn, Trevor

doi:10.18653/v1/p17-1033

Cited by 38 publications

(21 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the combination of deep learning and LDA has become another topic extraction method. For example, in this study [67], a novel neural topic model was proposed to acquire the N-gram topic by the deep neural network and then used LDA to obtain the topic representation of the document; the study [68] integrated LDA into language model LSTM for joint training. In addition to the LDA and its extensions, there are also methods for generating subject-related vectors by the neural attention model, such as Attention-based Aspect Extraction (ABAE) [69].…”

Section: Topic Extractionmentioning

confidence: 99%

A Review of Text Corpus-Based Tourism Big Data Mining

Zhang

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

With the massive growth of the Internet, text data has become one of the main formats of tourism big data. As an effective expression means of tourists' opinions, text mining of such data has big potential to inspire innovations for tourism practitioners. In the past decade, a variety of text mining techniques have been proposed and applied to tourism analysis to develop tourism value analysis models, build tourism recommendation systems, create tourist profiles, and make policies for supervising tourism markets. The successes of these techniques have been further boosted by the progress of natural language processing (NLP), machine learning, and deep learning. With the understanding of the complexity due to this diverse set of techniques and tourism text data sources, this work attempts to provide a detailed and up-, destination image analysis, market demand, etc. Our work also provides guidelines for constructing new tourism big data applications and outlines promising research areas in this field for incoming years. Appl. Sci. 2019, 9, 3300 2 of 27 the travel. In order to utilize this user-generated content properly and further to meet the needs of tourists and promote the tourism industry, we need to analyze and exploit tourists' needs and opinions, and then identify the problems of tourism services or destinations, which has become a new path for tourism development. Besides, as tourism needs become increasingly personalized, visitors begin to pursue self-likeness, self-worth, and diversified travel experiences, and they are no longer willing to endure delays or waits. How to recognize and respond to visitors' behaviors and needs quickly and identify potential customers have become essential factors for the success of tourism stakeholders. By exploiting the subjective information contained in tourism text data, we can assist tourism stakeholders to provide better services for tourists.A large number of text mining techniques have been proposed and applied to tourism text data analysis for creating tourist profiles [8][9][10][11][12][13][14][15] and making effective market supervision [16][17][18][19][20][21][22][23][24][25]. These approaches exploit a variety of text representation strategies [26][27][28][29][30][31][32] and use different NLP techniques for topic extraction [33], text classification [34], sentiment analysis [35], and text clustering [36]. Moreover, while aiming to make computers understand human language, NLP has become the essential tool for text data analysis and is undergoing fast-pace growing based on the applications of deep learning in word embedding, syntax analysis, machine translation, and text understanding. Machine learning-based NLP techniques have been widely used in tourism text analysis, with superior results [19,25]. In addition, due to its high capability for extracting selective and invariant features from texts, and its independency of prior knowledge and linguistic resources, deep learning has been reported to achieve higher performance than other approaches on many NL...

show abstract

Section: Topic Extractionmentioning

confidence: 99%

A Review of Text Corpus-Based Tourism Big Data Mining

Zhang

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Our work provides an additional way to address the wellknown drawback of RNNs: they use only limited context. This has been noted as a serious problem in conversational modeling (Sordoni et al, 2015) and text generation with multiple sentences (Lau et al, 2017). Recent work on context-aware text generation (or the related task, language modeling) has studied the possibilities of using different granularity of context.…”

Section: Related Workmentioning

confidence: 99%

Neural Text Generation in Stories Using Entity Representations as Context

Clark¹,

Ji²,

Smith³

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

109

View full text Add to dashboard Cite

We introduce an approach to neural text generation that explicitly represents entities mentioned in the text. Entity representations are vectors that are updated as the text proceeds; they are designed specifically for narrative text like fiction or news stories. Our experiments demonstrate that modeling entities offers a benefit in two automatic evaluations: mention generation (in which a model chooses which entity to mention next and which words to use in the mention) and selection between a correct next sentence and a distractor from later in the same story. We also conduct a human evaluation on automatically generated text in story contexts; this study supports our emphasis on entities and suggests directions for further research.

show abstract

“…• LSTM + LDA: the topic vector θ obtained from a pretrained LDA is concatenated to the output of LSTM and a recent topic-dependent LSTM applied to our task • TD-LSTM (Lau, Baldwin, and Cohn 2017): θ is added to the output of LSTM via a dense layer. TD-LSTM and our TE-LSTM are jointly trained with our Topic Generated Equations Quantum physics…”

Section: Equation Model Evaluationmentioning

confidence: 99%

TopicEq: A Joint Topic and Mathematical Equation Model for Scientific Texts

Yasunaga

Lafferty

2019

AAAI

View full text Add to dashboard Cite

Scientific documents rely on both mathematics and text to communicate ideas. Inspired by the topical correspondence between mathematical equations and word contexts observed in scientific texts, we propose a novel topic model that jointly generates mathematical equations and their surrounding text (TopicEq). Using an extension of the correlated topic model, the context is generated from a mixture of latent topics, and the equation is generated by an RNN that depends on the latent topic activations. To experiment with this model, we create a corpus of 400K equation-context pairs extracted from a range of scientific articles from arXiv, and fit the model using a variational autoencoder approach. Experimental results show that this joint model significantly outperforms existing topic models and equation models for scientific texts. Moreover, we qualitatively show that the model effectively captures the relationship between topics and mathematics, enabling novel applications such as topic-aware equation generation, equation topic inference, and topic-aware alignment of mathematical symbols and words.

show abstract

Topically Driven Neural Language Model

Cited by 38 publications

References 19 publications

A Review of Text Corpus-Based Tourism Big Data Mining

A Review of Text Corpus-Based Tourism Big Data Mining

Neural Text Generation in Stories Using Entity Representations as Context

TopicEq: A Joint Topic and Mathematical Equation Model for Scientific Texts

Contact Info

Product

Resources

About