Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems

Wen, Tao; Gašić, Milica; Mrkšić, Nikola; Su, Pei-Hao; Vandyke, David; Young, Steve

doi:10.18653/v1/d15-1199

Cited by 672 publications

(383 citation statements)

References 41 publications

Supporting

Mentioning

380

Contrasting

Unclassified

Order By: Relevance

“…Schocher et al [62] show that sentiment classification can be learned with deep learning models by using word embeddings as an input layer to a recursive neural network (RNN). In the field of natural language generation, deep learning has also been shown to be successful through the modelling of semantic constraints [63]. Ma et al [64] propose a deep learning approach for sentence embedding, and show that their method can achieve state-of-the-art results for many sentence classification problems such as sentiment classification and question classification.…”

Section: Future Trendsmentioning

confidence: 99%

Text analytics in industry: Challenges, desiderata and trends

Ittoo

Nguyen

Bosch

2016

Computers in Industry

View full text Add to dashboard Cite

Section: Future Trendsmentioning

confidence: 99%

Text analytics in industry: Challenges, desiderata and trends

Ittoo

Nguyen

Bosch

2016

Computers in Industry

View full text Add to dashboard Cite

“…However, in the case that no response in the database could adequately respond to a given utterance, this approach will fail. Response generation [15]- [17] which has the ability to generate a new responses, is arguably robust in handling user input comparing to the other approach, however this approach sometimes generates unnatural responses that are incomprehensible to the user [9]. There have been a number of works on response generation for data-driven dialog systems.…”

Section: Related Workmentioning

confidence: 99%

“…Sordoni et al [15] employ an RNN architecture to generate responses from a social media corpus, and Vinyals et al [16] present a long short-term memory (LSTM) neural network encoderdecoders to generate dialog responses using movie subtitles or IT support line chats. More recently Wen et al [17] demonstrate a more advanced LSTM that able to control a response semantically by considering dialogue act feature.…”

Section: Related Workmentioning

confidence: 99%

“…Generation models originally adapted statistical machine translation (SMT) to utilize a query-response dialog database as a parallel corpus to "translate" between query input and response output [9], [14]. In the recent developments, there have also been several alternative approaches developed simultaneously to this work that utilize neural networks for dialog generation [15]- [17].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neural Network Approaches to Dialog Response Retrieval and Generation

Nio

Sakti

Neubig

et al. 2016

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

Lasguido NIO†a) , Sakriani SAKTI †b) , Graham NEUBIG †c) , Koichiro YOSHINO †d) , Nonmembers, and Satoshi NAKAMURA †e) , Member SUMMARYIn this work, we propose a new statistical model for building robust dialog systems using neural networks to either retrieve or generate dialog response based on an existing data sources. In the retrieval task, we propose an approach that uses paraphrase identification during the retrieval process. This is done by employing recursive autoencoders and dynamic pooling to determine whether two sentences with arbitrary length have the same meaning. For both the generation and retrieval tasks, we propose a model using long short term memory (LSTM) neural networks that works by first using an LSTM encoder to read in the user's utterance into a continuous vector-space representation, then using an LSTM decoder to generate the most probable word sequence. An evaluation based on objective and subjective metrics shows that the new proposed approaches have the ability to deal with user inputs that are not well covered in the database compared to standard example-based dialog baselines. key words: example-based dialog system, dialog system, response retrieval, response generation, long short term memory neural network IntroductionNatural language dialogue systems promise to establish efficient interfaces for communication between humans and computers [1]- [5]. One way to create a simple yet effective dialog system is using example-based dialog modeling (EBDM) [6]-[9]. EBDM is a data-driven approach for creating dialog systems that choose how to respond to user input based on a large database of examples consisting of an utterance, and a corresponding natural reply to that utterance. Given a user input, the system then performs response retrieval, selecting the highest scoring response from the existing utterances in the database. EBDM presents a lightweight alternative to more conventional methods for constructing dialog systems, as it only requires the construction of an example base, and has also been shown effective in a number of dialog scenarios. In particular, this approach is able to generate highly natural output when an example that matches closely with the user query is included in the database and the example is appropriately retrieved [6] However, dealing with sparse human language and a finite query-response database, we can imagine easily that such system may fail when attempting to respond to a user utterance that does not match closely with one of the examples in the database. We define this kind of problem as an out of example (OOE) problem. One way to overcome this problem is using response generation. This approach uses the dialog examples as data to train a model that can generate responses not included in the database. Generation has the potential to be more robust to OOE user inputs, but also may generate responses that are incomprehensible to human users [13]. Generation models originally adapted statistical machine translation (SMT) to utilize a query-response dialog ...

show abstract

“…Natural Language Generation (NLG) is a challenging problem that has drawn a lot of attention in the Natural Language Processing (NLP) community [15,17]. NLG is crucial for multiple NLP applications and problems, such as dialogue systems [20], text summarization [4], and text paraphrasing [7]. Poem generation is an instance of NLG that is particularly fascinating for its peculiar features.…”

Section: Introductionmentioning

confidence: 99%

Artificial Neural Networks and Machine Learning – ICANN 2019: Text and Time Series

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Motivated by the recent progresses on machine learningbased models that learn artistic styles, in this paper we focus on the problem of poem generation. This is a challenging task in which the machine has to capture the linguistic features that strongly characterize a certain poet, as well as the semantics of the poet's production, that are influenced by his personal experiences and by his literary background. Since poetry is constructed using syllables, that regulate the form and structure of poems, we propose a syllable-based neural language model, and we describe a poem generation mechanism that is designed around the poet style, automatically selecting the most representative generations. The poetic work of a target author is usually not enough to successfully train modern deep neural networks, so we propose a multistage procedure that exploits non-poetic works of the same author, and also other publicly available huge corpora to learn syntax and grammar of the target language. We focus on the Italian poet Dante Alighieri, widely famous for his Divine Comedy. A quantitative and qualitative experimental analysis of the generated tercets is reported, where we included expert judges with strong background in humanistic studies. The generated tercets are frequently considered to be real by a generic population of judges, with relative difference of 56.25% with respect to the ones really authored by Dante, and expert judges perceived Dante's style and rhymes in the generated text.

show abstract

Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems

Cited by 672 publications

References 41 publications

Text analytics in industry: Challenges, desiderata and trends

Text analytics in industry: Challenges, desiderata and trends

Neural Network Approaches to Dialog Response Retrieval and Generation

Artificial Neural Networks and Machine Learning – ICANN 2019: Text and Time Series

Contact Info

Product

Resources

About