A Neural Network Approach to Context-Sensitive Generation of Conversational Responses

Sordoni, Alessandro; Galley, Michel; Auli, Michael; Brockett, Chris; Ji, Yangfeng; Nie, Jian‐Yun; Gao, Jianfeng; Dolan, Bill

doi:10.48550/arxiv.1506.06714

Cited by 88 publications

(135 citation statements)

References 0 publications

Supporting

Mentioning

135

Contrasting

Order By: Relevance

“…Our work is also closely related to recent successes in applying language models to dialog modeling (e.g., [25,26,17,18]), which built on earlier research in neural dialog modeling (e.g., [14,15,16,27,28]). One of our fine-tuning stages requires training on dialog-only data, which is related to Wolf et al [29], Dinan et al [25] and Zhang et al [30].…”

Section: Related Workmentioning

confidence: 82%

“…Our approach is inspired by Adiwardana et al [17], who argued for human-like metrics, such as sensibleness and specificity. Many automated metrics for dialog models have been studied, including perplexity [16,17], F1, Hits@1/N [25], USR [46], or BLEU/ROUGE [47,15,27]. However, such automated metrics may not correlate well with human judgment [48].…”

Section: Related Workmentioning

confidence: 99%

“…In Table 13, we show examples of generated responses from pre-trained and fine-tuned LaMDA models over contexts from the WoW dataset [96]. In Tables 14,15,16,17,18,19,20,21,22,23,24, 25 and 26 we show dialog examples collected by crowd-worker interaction with LaMDA, with hard-coded preconditioning messages in italic.…”

Section: Selected Example Dialogsmentioning

confidence: 99%

“…Dialog models [14,15,16], one of the most interesting applications of large language models, successfully take advantage of Transformers' ability to represent long-term dependencies in text [17,18]. Similar to general language models [13], Adiwardana et al [17] show that dialog models are also well suited to model scaling.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

LaMDA: Language Models for Dialog Applications

Thoppilan¹,

Freitas²,

Hall³

et al. 2022

Preprint

282

270

View full text Add to dashboard Cite

We present LaMDA: Language Models for Dialog Applications. LaMDA is a family of Transformerbased neural language models specialized for dialog, which have up to 137B parameters and are pre-trained on 1.56T words of public dialog data and web text. While model scaling alone can improve quality, it shows less improvements on safety and factual grounding. We demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding. The first challenge, safety, involves ensuring that the model's responses are consistent with a set of human values, such as preventing harmful suggestions and unfair bias. We quantify safety using a metric based on an illustrative set of human values, and we find that filtering candidate responses using a LaMDA classifier fine-tuned with a small amount of crowdworker-annotated data offers a promising approach to improving model safety. The second challenge, factual grounding, involves enabling the model to consult external knowledge sources, such as an information retrieval system, a language translator, and a calculator. We quantify factuality using a groundedness metric, and we find that our approach enables the model to generate responses grounded in known sources, rather than responses that merely sound plausible. Finally, we explore the use of LaMDA in the domains of education and content recommendations, and analyze their helpfulness and role consistency. * Work done while at Google.

show abstract

Section: Related Workmentioning

confidence: 82%

Section: Related Workmentioning

confidence: 99%

Section: Selected Example Dialogsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

LaMDA: Language Models for Dialog Applications

Thoppilan¹,

Freitas²,

Hall³

et al. 2022

Preprint

282

270

View full text Add to dashboard Cite

show abstract

“…The OpenSubtitle dataset is a large-scale dataset contains a total number of 3.35G sentence fragments extracted from the OpenSubtitle website, while the Cornell Movie-Dialogs Corpus contains a collection of movie conversations extracted from raw movie scripts. For simulating the social conversation, there are PersonaChat (Zhang et al, 2018) and Twitter Triple Corpus (Sordoni et al, 2015). The Twitter Triple Corpus consists of 4,232 Twitter conversation triples evaluated from 33K candidate triples by human raters.…”

Section: Introductionmentioning

confidence: 99%

OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts

Wang¹,

Meng²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

In order to better simulate the real human conversation process, models need to generate dialogue utterances based on not only preceding textual contexts but also visual contexts. However, with the development of multi-modal dialogue learning, the dataset scale gradually becomes a bottleneck. In this report, we release OpenViDial 2.0, a larger-scale open-domain multi-modal dialogue dataset compared to the previous version OpenViDial 1.0 (Meng et al., 2020). OpenViDial 2.0 contains a total number of 5.6 million dialogue turns extracted from either movies or TV series from different resources, and each dialogue turn is paired with its corresponding visual context. We hope this large-scale dataset can help facilitate future researches on open-domain multi-modal dialog generation, e.g., multi-modal pretraining for dialogue generation. 1

show abstract

Data-Driven Dialogue Systems for Social Agents

Bowden

Oraby

Misra

et al. 2018

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

In order to build dialogue systems to tackle the ambitious task of holding social conversations, we argue that we need a data-driven approach that includes insight into human conversational "chit-chat", and which incorporates different natural language processing modules. Our strategy is to analyze and index large corpora of social media data, including Twitter conversations, online debates, dialogues between friends, and blog posts, and then to couple this data retrieval with modules that perform tasks such as sentiment and style analysis, topic modeling, and summarization. We aim for personal assistants that can learn more nuanced human language, and to grow from task-oriented agents to more personable social bots.

show abstract

A Neural Network Approach to Context-Sensitive Generation of Conversational Responses

Cited by 88 publications

References 0 publications

LaMDA: Language Models for Dialog Applications

LaMDA: Language Models for Dialog Applications

OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts

Data-Driven Dialogue Systems for Social Agents

Contact Info

Product

Resources

About