Enhancing generative conversational service agents with dialog history and external knowledge

Wang, Zongsheng; Wang, Zhuoran; Long, Yinong; Wang, Jianan; Xu, Zhen; Wang, Baoxun

doi:10.1016/j.csl.2018.09.003

Cited by 22 publications

(16 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[2]'s idea of using additional topics encoder was later adopted and enhanced in multi-task learning set up by [61] to generate a more topic coherent and diverse sentences. [34], [35], [62], [63] proposed additional facts encoder to encode and inject facts retrieved based on keywords in the input sentence to the decoder.…”

Section: ) Additional Embeddingsmentioning

confidence: 99%

Enhancements to the Sequence-to-Sequence-Based Natural Answer Generation Models

et al. 2020

View full text Add to dashboard Cite

There is a great interest shown by academic researchers to continuously improve the sequence-to-sequence (Seq2Seq) model for natural answer generation (NAG) in chatbots. The Seq2Seq model shows a weakness whereby the model tends to generate answers that are generic, meaningless and inconsistent with the questions. However, a comprehensive literature review on the factors contributing to the weakness and potential solutions are still missing. Therefore, this review article fills the gap by reviewing Seq2Seq based natural answer generation-based literature to identify those factors and proposed methods to address the weakness. This literature review identified several factors such as input question is not sufficient to determine a meaningful output, usage of cross-entropy function as the loss function during training, infrequent words in training data, language model influence which generates answers not relevant to the question, utilization of teacher forcing method during training which results in exposure bias, long sentences and inability to consider dialogue history as the factors. Additionally, this literature review also identified and reviewed the methods proposed to address the weakness such as utilizing additional embedding and encoders, using different loss functions and training approaches, as well as utilizing other mechanisms like copying source word(s) and paying attention to a certain portion of the input. For discussion, these methods are categorized into four broad categories which are Structural Modifications, Augmented Learning, Beam Search and Complementary Mechanisms. Additionally, the paper highlights unexplored areas in Seq2Seq modeling and proposes potential future works for natural answer generation. INDEX TERMS Seq2Seq, natural answer generation, natural language processing, dialogue generation, chatbot.

show abstract

Section: ) Additional Embeddingsmentioning

confidence: 99%

Enhancements to the Sequence-to-Sequence-Based Natural Answer Generation Models

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The generative Chatbot is built in order to be able to act to any context or interlocutor and also in nonprogrammed situations. Such a conversational agent relies on the new artificial intelligence techniques such as deep learning and neural network to generate his responses word by word [8]. Thus, these bots can construct answers to users questions themselves.…”

Section: A Chatbotmentioning

confidence: 99%

Adaptive e-Learning AI-Powered Chatbot based on Multimedia Indexing

Janati¹,

Maach²,

Ghanami³

2020

IJACSA

View full text Add to dashboard Cite

“…This raises the prospect of building a conversational system using a data-driven approach for open domain topics. Existing methods along this line include retrieval-based methods [19]- [22], which search a large corpus and select the response that best matches the context, and generation-based methods [2], [10], [14], [23], [24], which aim to learn a response generation model from conversations to generate a proper response for a new context. In this work, our study is focused on generation-based methods.…”

Section: Related Work a Conversation Systemsmentioning

confidence: 99%

A Hierarchical Structured Multi-Head Attention Network for Multi-Turn Response Generation

et al. 2020

View full text Add to dashboard Cite

As a crucial task in conversation systems, response generation for multi-turn conversation aims to generate a coherent, informative and diverse response according to the conversation context. Existing models of this task are limited in their ability to capture long-term dependencies within and between utterances and to identify pertinent or important information in the context. Inspired by the Transformer neural network based solely on attention mechanisms recently proposed in machine translation, we propose a novel hierarchical structured multi-head attention network (HMAN) model to address both problems. Specifically, the context sequences are encoded in a hierarchical structure in which multi-head self-attention is first employed to compute the representation matrix of each utterance, and then, these utterance matrices are integrated to form a context representation with the complicated dependencies learned using multi-head attention. The experimental results on two public conversation datasets demonstrate that our proposed model significantly outperforms several state-of-the-art baselines with respect to both automatic evaluation and human evaluation and can generate more diverse and informative responses.

show abstract

Enhancing generative conversational service agents with dialog history and external knowledge

Cited by 22 publications

References 1 publication

Enhancements to the Sequence-to-Sequence-Based Natural Answer Generation Models

Enhancements to the Sequence-to-Sequence-Based Natural Answer Generation Models

Adaptive e-Learning AI-Powered Chatbot based on Multimedia Indexing

A Hierarchical Structured Multi-Head Attention Network for Multi-Turn Response Generation

Contact Info

Product

Resources

About